Pathom Tutorial - IP Weather

The task of this tutorial is to implement a series of resolvers that can tell the current temperature, based on some IP address.

To implement this, I'll use GeoJS to find the location from some given IP, and then use Meta Weather to find the temperature.

App Setup

In this demo, I'll be using Clojure Deps to manage the dependencies:

deps.edn
{:paths
["src"]
:deps
{cheshire/cheshire {:mvn/version "5.10.0"}
com.wsscode/pathom3 {:git/url "https://github.com/wilkerlucio/pathom3"
:sha "d2046cec34b75a2ac70d3125276394b68a34968a"}
org.clojure/clojure {:mvn/version "1.10.1"}}}

We are going to use cheshire to parse JSON from the API responses.

Command Line Application

To run our application, we will use the :exec-fn feature from deps.edn, I'll start setting up the main entry point and show how to trigger it from the command line:

(ns com.wsscode.pathom3.demos.ip-weather)
(defn main [{:keys [ip]}]
(println "Request temperature for the IP" ip))

To test this, run the following:

clj -X com.wsscode.pathom3.demos.ip-weather/main :ip '"198.29.213.3"'

To make this command shorter, add an alias to deps.edn:

deps.edn
{:paths
["src"]
:deps
{cheshire/cheshire {:mvn/version "5.10.0"}
com.wsscode/pathom3 {:git/url "https://github.com/wilkerlucio/pathom3"
:sha "1f3ca76ead855609e0f27b30f6e8bf23b5bcfa0a"}
org.clojure/clojure {:mvn/version "1.10.0"}}
; add alias to make easier to call
:aliases
{:ip-weather
{:exec-fn com.wsscode.pathom3.demos.ip-weather/main}}}

Now we can run:

clj -X:ip-weather :ip '"198.29.213.3"'

Scaffolding is done. Time to start writing some application logic.

Start from the tail

First, let's understand the data scenario for this task. If we start assuming that we know nothing about the services involved, we still know what we have (the IP) and what we want (the temperature), we can start with this graph representation:

By looking at the documentation on the Meta Weather API, I see that the temperature information is present in the https://www.metaweather.com/api/location/$WOEID$ endpoint (WOEID stands for Where On Earth ID) , under consolidated_weather -> the_temp. This means that to fetch the temperature, we need some woeid.

Meta Weather provides search endpoints to figure out the WOEID:

https://www.metaweather.com/api/location/search/?query=san
https://www.metaweather.com/api/location/search/?query=london
https://www.metaweather.com/api/location/search/?lattlong=36.96,-122.02
https://www.metaweather.com/api/location/search/?lattlong=50.068,-5.316

To make this example more interesting, we are going to use the one with latitude and longitude. In these systems, this means that now WOEID depends on latitude and longitude:

Finally, using the GeoJS API, we can use the endpoint https://get.geojs.io/v1/ip/geo/$IP$.json to figure the latitude and longitude, given some IP:

Now we have the complete path from the IP to the temperature.

tip

I found those API's using the Public API's service, I find it a great source to look for open API's to play.

Resolvers

In Pathom, resolvers are the main building blocks express attribute relationships.

To implement the resolvers, I'll start with the one to fetch the latitude and longitude from the IP:

(ns com.wsscode.pathom3.demos.ip-weather
(:require
[cheshire.core :as json]
[com.wsscode.pathom3.connect.operation :as pco]))
(pco/defresolver ip->lat-long
[{:keys [ip]}]
{::pco/output [:latitude :longitude]}
(-> (slurp (str "https://get.geojs.io/v1/ip/geo/" ip ".json"))
(json/parse-string keyword)
(select-keys [:latitude :longitude])))
(defn main [{:keys [ip]}]
(println "Request temperature for the IP" ip))

A resolver is like a function, with some constraints:

  1. The resolver input must be a map, so the input information is labeled.
  2. A resolver must return a map, so the output information is labeled.
  3. A resolver may also receive another map containing the environment information.

To test the resolver in the REPL, call it like a function:

(ip->lat-long {:ip "198.29.213.3"})
; => {:longitude "-88.0569", :latitude "41.5119"}

A resolver is a custom type, here is what's inside:

ip->lat-long
#com.wsscode.pathom3.connect.operation.Resolver
{:config
#:com.wsscode.pathom3.connect.operation
{:input [:ip]
:provides {:longitude {}
:latitude {}}
:output [:longitude
:latitude]
:op-name com.wsscode.pathom3.demos.ip-weather/ip->lat-long},
:resolve
#object[com.wsscode.pathom3.demos.ip_weather$ip__GT_lat_long__17350
0x4b7b5266
"com.wsscode.pathom3.demos.ip_weather$ip__GT_lat_long__17350@4b7b5266"]}

Note that in the configuration map of the resolver, we have the same ::pco/output as we wrote in the resolver, while the ::pco/input was inferred from the destructuring used in the resolver attribute vector.

You can learn more about the details at resolvers documentation page.

Now that we have the latitude and longitude, the next resolver will find a WOEID from that:

(ns com.wsscode.pathom3.demos.ip-weather
(:require
[cheshire.core :as json]
[com.wsscode.pathom3.connect.operation :as pco]))
(pco/defresolver ip->lat-long
[{:keys [ip]}]
{::pco/output [:latitude :longitude]}
(-> (slurp (str "https://get.geojs.io/v1/ip/geo/" ip ".json"))
(json/parse-string keyword)
(select-keys [:latitude :longitude])))
(pco/defresolver latlong->woeid
[{:keys [latitude longitude]}]
{:woeid
(-> (slurp
(str "https://www.metaweather.com/api/location/search/?lattlong="
latitude "," longitude))
(json/parse-string keyword)
first
:woeid)})
(defn main [{:keys [ip]}]
(println "Request temperature for the IP" ip))
note

In latlong->woeid resolver, we let Pathom infer the output automatically. To use this feature, remember that the last expression must be a map. Otherwise, Pathom will not try to infer the output.

Testing the resolver in the REPL:

(latlong->woeid {:longitude "-88.0569", :latitude "41.5119"})
; => {:woeid 2379574}

We are getting close, the final step is to find out the temperature, given the WOEID:

(ns com.wsscode.pathom3.demos.ip-weather
(:require
[cheshire.core :as json]
[com.wsscode.pathom3.connect.operation :as pco]))
(pco/defresolver ip->lat-long
[{:keys [ip]}]
{::pco/output [:latitude :longitude]}
(-> (slurp (str "https://get.geojs.io/v1/ip/geo/" ip ".json"))
(json/parse-string keyword)
(select-keys [:latitude :longitude])))
(pco/defresolver latlong->woeid
[{:keys [latitude longitude]}]
{:woeid
(-> (slurp
(str "https://www.metaweather.com/api/location/search/?lattlong="
latitude "," longitude))
(json/parse-string keyword)
first
:woeid)})
(pco/defresolver woeid->temperature
[{:keys [woeid]}]
{:temperature
(-> (slurp (str "https://www.metaweather.com/api/location/" woeid))
(json/parse-string keyword)
:consolidated_weather
first
:the_temp)})
(defn main [{:keys [ip]}]
(println "Request temperature for the IP" ip))

To keep our REPL testing in check:

(woeid->temperature {:woeid 2379574})
; => {:temperature 4.529999999999999}

The whole process chains nicely, starting from ip to temperature, like this:

(-> {:ip "198.29.213.3"}
ip->lat-long
latlong->woeid
woeid->temperature)
; => {:temperature 4.529999999999999}

Graph Traversal

In the previous example, we were able to find the temperature starting from the IP. I like to point all the names involved in the operation when we finished it, let's look at it again:

(-> {:ip "198.29.213.3"}
ip->lat-long
latlong->woeid
woeid->temperature)

We have the :ip attribute in a map, and then we have 3 function names, which dictates the step. Now we will to replace all the resolver names with a single attribute: our data demand, the :temperature.

It's time to leverage the attribute relations established from the resolvers.

To do this, Pathom needs some indexes that combine the attribute relations described by a list of resolvers. This is demonstrated in the highlighted fragments of the following snippet:

(ns com.wsscode.pathom3.demos.ip-weather
(:require
[cheshire.core :as json]
[com.wsscode.pathom3.connect.indexes :as pci]
[com.wsscode.pathom3.connect.operation :as pco]))
(pco/defresolver ip->lat-long
[{:keys [ip]}]
{::pco/output [:latitude :longitude]}
(-> (slurp (str "https://get.geojs.io/v1/ip/geo/" ip ".json"))
(json/parse-string keyword)
(select-keys [:latitude :longitude])))
(pco/defresolver latlong->woeid
[{:keys [latitude longitude]}]
{:woeid
(-> (slurp
(str "https://www.metaweather.com/api/location/search/?lattlong="
latitude "," longitude))
(json/parse-string keyword)
first
:woeid)})
(pco/defresolver woeid->temperature
[{:keys [woeid]}]
{:temperature
(-> (slurp (str "https://www.metaweather.com/api/location/" woeid))
(json/parse-string keyword)
:consolidated_weather
first
:the_temp)})
(def env
(pci/register [ip->lat-long
latlong->woeid
woeid->temperature]))
(defn main [{:keys [ip]}]
(println "Request temperature for the IP" ip))

Using the indexes we generated at the name of env, we can do the same processing without mentioning any resolver name, using a Smart Map:

(-> (psm/smart-map env {:ip "198.29.213.3"})
:temperature)
; 5.24

Note the previous snippet doesn't include the name of any resolver!

We can move that code to our main function and make our program work:

(ns com.wsscode.pathom3.demos.ip-weather
(:require
[cheshire.core :as json]
[com.wsscode.pathom3.connect.indexes :as pci]
[com.wsscode.pathom3.connect.operation :as pco]
[com.wsscode.pathom3.interface.smart-map :as psm]))
(pco/defresolver ip->lat-long
[{:keys [ip]}]
{::pco/output [:latitude :longitude]}
(-> (slurp (str "https://get.geojs.io/v1/ip/geo/" ip ".json"))
(json/parse-string keyword)
(select-keys [:latitude :longitude])))
(pco/defresolver latlong->woeid
[{:keys [latitude longitude]}]
{:woeid
(-> (slurp
(str "https://www.metaweather.com/api/location/search/?lattlong="
latitude "," longitude))
(json/parse-string keyword)
first
:woeid)})
(pco/defresolver woeid->temperature
[{:keys [woeid]}]
{:temperature
(-> (slurp (str "https://www.metaweather.com/api/location/" woeid))
(json/parse-string keyword)
:consolidated_weather
first
:the_temp)})
(def env
(pci/register [ip->lat-long
latlong->woeid
woeid->temperature]))
(defn main [args]
; start smart maps with call args
(let [sm (psm/smart-map env args)]
(println (str "It's currently " (:temperature sm) "C at " (pr-str args)))))

Then we can run from the command line:

# some specific IP
clj -X:ip-weather :ip '"198.29.213.3"'
# => It's currently 8.33C at {:ip "198.29.213.3"}
# get from your IP
clj -X:ip-weather :ip "\"$(curl -s ifconfig.me)\""
# => It's currently ??C at {:ip "YOUR_IP"}

Magic? No, it's the power of graphs!

To help understand how this works, have a look inside that env variable we defined (the map on the right side):

note

For now, let's focus on the index-oir, which is the main index used to traverse dependencies. Check the indexes page to learn more about the other indexes.

When we request the :temperature, Pathom looks in the index for a path to that attribute. It depends on :woeid, which we don't have, but the index says you can get it if you provide :latitude and :longitude. We don't have those either, then Pathom has to look again in the index for then, and those are available through :ip, which is in the data context. It's the same path we described before with the table column associations.

The previous paragraph described the planning process. The output of that process is an execution graph that describes what will take to fulfill the demand. Then it starts running it, first figure :latitude and :longitude from :ip:

With :latitude and :longitude available, now the process can fetch the :woeid:

Finally, use :woeid to get the :temperature:

Because our code only talks about context and demand, we can also use this command line tool in a few other forms:

# from lat long
clj -X:ip-weather :latitude '"41.5119"' :longitude '"-88.0569"'
# => It's currently 8.33C at {:latitude "41.5119", :longitude "-88.0569"}
# from woeid
clj -X:ip-weather :woeid 2379574
# => It's currently 8.33C at {:woeid 2379574}

As long as you use some data that has a path to the temperature, it works.

What's next

This concludes this tutorial. A quick review:

  • Map the available data you have and the data you want in terms of attributes.
  • Write resolvers connecting the attribute names, adding more attributes as needed.
  • Prepare an environment with the indexes.
  • Use EQL to make the information request.

I designed this demo to illustrate the basic concepts of attribute modeling and Pathom.

Here are a few exercise suggestions you can do to extend this demo:

  • Add a resolver to tell if the temperature is cold or not, based on some cold threshold
  • Add a resolver to use the Meta Weather API to search based on a search query.
  • Add a resolver to use the current user public IP when nothing else is provided.