Smart Maps
Smart maps are a custom map type powered by Pathom resolvers.
With smart maps, you can leverage the power of Pathom processing using the accessible standard Clojure map interfaces.
Using smart maps
To create a smart map, we need the indexes with resolvers, here is a basic example:
(ns com.wsscode.pathom.docs.smart-maps-demo
(:require [com.wsscode.pathom3.connect.indexes :as pci]
[com.wsscode.pathom3.connect.operation :as pco]
[com.wsscode.pathom3.interface.smart-map :as psm]))
(pco/defresolver full-name [{::keys [first-name last-name]}]
{::full-name (str first-name " " last-name)})
(def indexes (pci/register full-name))
(def person-data {::first-name "Anne" ::last-name "Frank"})
(def smart-map (psm/smart-map indexes person-data))
; if you lookup for a key in the initial data, it works the same way as a regular map
(::first-name smart-map) ; => "Anne"
; but when you read something that's not there, it will trigger the Pathom engine to
; fulfill the attribute
(::full-name smart-map) ; => "Anne Frank"
When you start a smart map, Pathom creates an atom with the initial data, and then when you request some new information, Pathom triggers the resolver engine, merges the result in the same atom and returns the value for that key, effectively caching it.
This way, the subsequent accesses have the same speed as a local entry.
Nested maps
Nested map values are wrapped with a smart map using the same environment configuration. For example:
(pco/defresolver full-name [{::keys [first-name last-name]}]
{::full-name (str first-name " " last-name)})
(pco/defresolver anne []
{::anne {::first-name "Anne" ::last-name "Frank"}})
(def indexes (pci/register [full-name anne]))
(def smart-map (psm/smart-map indexes))
(::anne smart-map) ; => {::first-name "Anne" ::last-name "Frank"}
; nested access
(-> smart-map ::anne ::full-name) ; => "Anne Frank"
This only applies to native Clojure/script maps. It doesn't wrap records and other custom map types.
Nested sequences
It also applies for maps inside sequences:
(pco/defresolver full-name [{::keys [first-name last-name]}]
{::full-name (str first-name " " last-name)})
(pco/defresolver stars []
{::star-wars-characters
[{::first-name "Luke" ::last-name "Skywalker"}
{::first-name "Darth" ::last-name "Vader"}
{::first-name "Han" ::last-name "Solo"}]})
(def indexes (pci/register [full-name stars]))
(def smart-map (psm/smart-map indexes))
; nested access on sequences
(mapv ::full-name (::star-wars-characters smart-map))
; => ["Luke Skywalker"
; "Darth Vader"
; "Han Solo"]
Smart maps do the conversion of sequence items to smart maps on the sequence read. This
means that anytime you read a sequence on a smart map, if the sequence is a vector, that
scan will be done eagerly. Otherwise, it will use map, and the processing is lazy.
Error modes
By default, if some error happens during the Pathom process, the Smart Map is going
to be silent about it. You can allow the errors to flow up by using the psm/with-error-mode
helper:
(pco/defresolver error-resolver []
{:error (throw (ex-info "Error" {}))})
(let [sm (-> (pci/register error-resolver)
(psm/with-error-mode ::psm/error-mode-loud)
(psm/smart-map))]
(:error sm))
; => Execution error (ExceptionInfo) at ...
; Error
The options for error mode are:
::psm/error-mode-silent(default): Returnnilas the value, don't throw errors.::psm/error-mode-loud: Throw the errors on read.
Disable nested wrap
You can disable the automatic nest wrap using (psm/with-wrap-nested? false) in the env.
(psm/smart-map (-> indexes
(psm/with-wrap-nested? false)
{:initial-data "value"}))
Then all values will return as-is.
You can manually wrap nested values in the same way the library does by using the same env on the value, for example:
(psm/smart-map (psm/sm-env smart-map) (::map-value smart-map))
Preload data
If you know the attributes you will need ahead of time, it's more efficient to load then in a single run than fetching one by one lazily.
You can accomplish this using the fn psm/touch!, example:
(pco/defresolver right [{::keys [left width]}]
{::right (+ left width)})
(pco/defresolver bottom [{::keys [top height]}]
{::bottom (+ top height)})
(def indexes (pci/register [right bottom]))
(def square {::left 10 ::top 30
::width 23 ::height 35})
(def smart-map
(-> (psm/smart-map indexes square)
(psm/sm-touch! [::right ::bottom])))
(::right smart-map) ; => 33, read from cache
(::bottom smart-map) ; => 65, read from cache
You can also use psm/sm-touch-ast! to provide an AST directly.
Keys modes
Smart Maps keys mode is a configuration to decide how a smart map respond to (keys smart-map)
To change this, use the helper (psm/with-keys-mode ...), these are the available options
for it:
::psm/keys-mode-cached- the default option,keyswill return the keys cached in the internal smart map atom.::psm/keys-mode-reachable-keyswill return all possible keys that are reachable from the current data and the index.
Be careful with ::psm/keys-mode-reachable combined with enabled "nested wrapping".
Considering that, depending on the index and the current data, a simple print of the
smart map can lead to infinite loops due to smart maps' recursive properties.
Changes to smart maps
You can use the change operations of maps in the smart map (assoc, dissoc, ...).
When a change operation happens, you get a new smart map. Be aware this new smart map
doesn't have the cached data from resolvers from the previous one. The modification is
done from the source map, the one used to create the smart map in the first place.
This is important so any data computed by resolvers can react to the changes based on the new data context. The following example illustrates this behavior:
(pco/defresolver full-name [{::keys [first-name last-name]}]
{::full-name (str first-name " " last-name)})
(def indexes (pci/register full-name))
(def person-data {::first-name "John" ::last-name "Lock"})
(def smart-map (psm/smart-map indexes person-data))
(::full-name smart-map) ; => "John Lock"
(-> smart-map
(assoc ::last-name "Oliver")
::full-name)
; => "John Oliver", the full-name gets re-computed due to the change
Inside smart maps
The smart map data structure contains an environment inside of it, there you can find the indexes, the cached data, and any other relevant options that make the setup of the smart map.
You can access the smart map environment using psm/sm-env. This includes all the data
you sent as env during the smart map creation, plus the cache atom.
Debugging reads
Sometimes the result will be unexpected, to debug the smart map you can use the psm/sm-get-with-stats
function to return the run stats of the process:
(def indexes
(pci/register
[(pbir/constantly-resolver ::n 10)
(pbir/single-attr-resolver ::n ::x inc)
(pbir/single-attr-resolver ::x ::y #(* % 2))
(pbir/single-attr-resolver ::y ::z #(- % 10))]))
(-> (psm/smart-map indexes)
(psm/sm-get-with-stats ::y))
;{:com.wsscode.pathom3.connect.planner/index-attrs #:com.wsscode.pathom.docs.smart-maps-demo{:n #{3},
; :y #{1},
; :x #{2}},
; :com.wsscode.pathom3.connect.planner/unreachable-attrs {},
; :com.wsscode.pathom3.connect.runner/graph-process-duration-ms 0.11225199699401855,
; :com.wsscode.pathom3.connect.runner.stats/overhead-duration-ms 0.09637504816055298,
; :com.wsscode.pathom3.connect.planner/root 3,
; :com.wsscode.pathom3.connect.runner.stats/overhead-duration-percentage 0.8585597650052356,
; :com.wsscode.pathom3.connect.runner/node-run-stats {1 #:com.wsscode.pathom3.connect.runner{:run-duration-ms 0.006672978401184082,
; :node-run-input #:com.wsscode.pathom.docs.smart-maps-demo{:x 11}},
; 3 #:com.wsscode.pathom3.connect.runner{:run-duration-ms 0.004477977752685547,
; :node-run-input {}},
; 2 #:com.wsscode.pathom3.connect.runner{:run-duration-ms 0.004725992679595947,
; :node-run-input #:com.wsscode.pathom.docs.smart-maps-demo{:n 10}}},
; :com.wsscode.pathom3.connect.planner/index-ast #:com.wsscode.pathom.docs.smart-maps-demo{:y {:key :com.wsscode.pathom.docs.smart-maps-demo/y,
; :type :prop,
; :dispatch-key :com.wsscode.pathom.docs.smart-maps-demo/y}},
; :com.wsscode.pathom3.connect.planner/unreachable-resolvers #{},
; :com.wsscode.pathom3.interface.smart-map/value 22,
; :com.wsscode.pathom3.connect.planner/index-resolver->nodes {com.wsscode.pathom.docs.smart_maps_demo_SLASH_x->com.wsscode.pathom.docs.smart_maps_demo_SLASH_y-single-attr-transform #{1},
; com.wsscode.pathom.docs.smart_maps_demo_SLASH_n-constant #{3},
; com.wsscode.pathom.docs.smart_maps_demo_SLASH_n->com.wsscode.pathom.docs.smart_maps_demo_SLASH_x-single-attr-transform #{2}},
; :com.wsscode.pathom3.connect.planner/nodes {1 {:com.wsscode.pathom3.connect.planner/after-nodes #{2},
; :com.wsscode.pathom3.connect.planner/requires #:com.wsscode.pathom.docs.smart-maps-demo{:y {}},
; :com.wsscode.pathom3.connect.operation/op-name com.wsscode.pathom.docs.smart_maps_demo_SLASH_x->com.wsscode.pathom.docs.smart_maps_demo_SLASH_y-single-attr-transform,
; :com.wsscode.pathom3.connect.planner/source-for-attrs #{:com.wsscode.pathom.docs.smart-maps-demo/y},
; :com.wsscode.pathom3.connect.planner/input #:com.wsscode.pathom.docs.smart-maps-demo{:x {}},
; :com.wsscode.pathom3.connect.planner/node-id 1},
; 3 {:com.wsscode.pathom3.connect.planner/requires #:com.wsscode.pathom.docs.smart-maps-demo{:n {}},
; :com.wsscode.pathom3.connect.operation/op-name com.wsscode.pathom.docs.smart_maps_demo_SLASH_n-constant,
; :com.wsscode.pathom3.connect.planner/source-for-attrs #{:com.wsscode.pathom.docs.smart-maps-demo/n},
; :com.wsscode.pathom3.connect.planner/input {},
; :com.wsscode.pathom3.connect.planner/run-next 2,
; :com.wsscode.pathom3.connect.planner/node-id 3},
; 2 {:com.wsscode.pathom3.connect.planner/after-nodes #{3},
; :com.wsscode.pathom3.connect.planner/requires #:com.wsscode.pathom.docs.smart-maps-demo{:x {}},
; :com.wsscode.pathom3.connect.operation/op-name com.wsscode.pathom.docs.smart_maps_demo_SLASH_n->com.wsscode.pathom.docs.smart_maps_demo_SLASH_x-single-attr-transform,
; :com.wsscode.pathom3.connect.planner/source-for-attrs #{:com.wsscode.pathom.docs.smart-maps-demo/x},
; :com.wsscode.pathom3.connect.planner/input #:com.wsscode.pathom.docs.smart-maps-demo{:n {}},
; :com.wsscode.pathom3.connect.planner/run-next 1,
; :com.wsscode.pathom3.connect.planner/node-id 2}},
; :com.wsscode.pathom3.connect.runner.stats/resolver-accumulated-duration-ms 0.015876948833465576}
Smart Maps caching
Smart maps use a durable form of the cache by default. The smart maps add the caches at initialization, and they are persisted when variations of that Smart Map are created.
This means, for example, when you assoc on a Smart Map, the new returned Smart Map still shares the same plan and resolver caches.
Datafy
Smart Maps support the Clojure Datafy / Navigate protocols. This means if you use a REPL visualizer like Reveal or REBL you can navigate the projected data from the Smart Map lazily.
To demonstrate I'll be showing its usage with Reveal in the following video:
Also in REBL: