Adding Live Weather to Your Grafana Home Dashboard

Back in Part 4 of Trending & Analyzing SmartThings Devices we created a Grafana dashboard to display details of devices via SmartThings. We'll build on this dashboard over time, and an important theme here is the unification of data from multiple services. SmartThings natively integrates with AccuWeather to report current temperature and the like, but what if you really want to see a snapshot of the week's forecast on your home dashboard?

DarkSky offers one of the best developer APIs around. I'm not paid to say that, they're not sponsoring this post (they don't even know about it), but having worked with their API on several other projects I've always been super impressed with their data models and the specificity of the data they return. Instead of basing their data on a large area (like a zip code), DarkSky uses your specific latitude and longitude, giving you a very precise forecast. We won't be directly using the API in this particular post, but if you're curious you should definitely go check it out.

In this particular case, we'll simply be adding DarkSky's embed widget to the Grafana dashboard we made previously.

DarkSky provides information on the embed process on their own blog, but I'll also document the entire end-to-end process here for completeness. We'll start by adding a "Text" panel to our Grafana dashboard. Click on the Panel Title of your new text panel, and edit it. You'll initially be taken to the "Options" tab, and you'll want set the "Mode" to "HTML". Paste the following script into the box, being sure to set your own Latitude and Longitude in place of the NN.NNNNNN's in line 10.

There's a few things going on in this script, but it's all pretty straight forward. Instead of simply adding the iframe from the DarkSky blog post, we're using JavaScript to populate the source of an empty iframe. The reason for this is so that we can refresh the iframe automatically without having to manually interact with the browser. Line 16 resets the source of the iframe every 1,800,000 milliseconds (or 30 minutes).

While I'm sharing this solution publicly, please be aware that DarkSky provides this embed functionality for free and does not charge for the API usage that powers it. Please be respectful of this and don't abuse it by using an overly high refresh rate.

Once you've pasted the code in and plugged in your latitude and longitude, switch over to the General tab. Here I've set a height of 120px, and erased the title so that there is no panel title in the weather widget on my dashboard.

Close the panel configuration by clicking the "X" to the right of the tabs to go back to your dashboard and admire your work. As with previous examples, the full JSON for this panel is available in my Grafana repo.

Trending & Analyzing SmartThings Devices (Part 4 of 4)

Table of Contents

What is Grafana

Grafana is an analytics platform that allows you to create rich dashboards, define alerting logic when values exceed specified ranges, and visualize your data across a number of different formats. You can get a good idea of the capabilities of Grafana by just clicking through the official and community submitted dashboards on the Grafana website. The beauty of Grafana is its ability to visualize incredibly complex data sets in a way that still looks really sexy.

Getting Started

Much like the installation of InfluxDB back in Part 3, the installation of Grafana is pretty straight forward. If you're using a separate server for Grafana and haven't performed the steps on it from Part 2, make sure your OS is up to date. I'm using Ubuntu 16.04, so running a sudo apt-get update && sudo apt-get upgrade && sudo reboot does the trick. If you're using a local device (such as a virtual machine, Raspberry Pi or Pine64) you're also going to want to make sure you have a static IP set.

Installing Grafana

Before we start the installation, we need to install the libfontconfig package, as Grafana depends on this. If you forget to do this, your Grafana install will simply fail and you'll have to come back and do it.

sudo apt-get install libfontconfig

Installation starts by grabbing the download url from Grafana's website for your OS, then downloading and installing Grafana with the supplied commands.

wget https://s3-us-west-2.amazonaws.com/grafana-releases/release/grafana_4.4.2_amd64.deb

sudo dpkg -i grafana_4.4.2_amd64.deb

With the packages installed, we can start Grafana by simply running sudo service grafana-server start.

We'll also want to ensure that Grafana starts automatically with our server by running sudo update-rc.d grafana-server defaults 95 10

We can now access our fresh Grafana installation by opening a web browser to http://{server-ip}:3000.

Configuring Grafana

Because I'm running Grafana inside my own network, protected by my own firewall, I want to enable anonymous access. This will allow me to easily run my Grafana dashboard on a computer in my office so I can quickly and easily glance at it. If your server is publicly accessible you'll undoubtedly want to keep it secured and take some additional steps to harden your server.

Grafana's configuration file is located at /etc/grafana/grafana.ini, so we'll edit it by running sudo nano /etc/grafana/grafana.ini.

Find the "Anonymous Auth" section, and set the enabled flag to true to allow anonymous access:

;enabled = true

Note that this configuration does not completely open up Grafana. You still need to login with either the default admin account or another admin account you create in order to administer dashboards. Anonymous access simply allows users access to dashboards directly as viewers.

Save the grafana.ini file by exiting Nano (Ctrl + X), and confirm that you want to save your changes. You now must restart Grafana for the new settings to take effect by running sudo service grafana-server restart.

Once the service has been restarted, point your web browser to http://{server-ip}:3000. You'll be prompted to login, and can use the default admin/admin credentials.

The last step required to configuration is to change the admin password. Grafana doesn't force you to do this, but it's always a best practice to change a default password when you first install something.

Click on the Grafana logo at the top left corner, and navigate down to admin > profile.

Scroll to the bottom of the page and "Change Password".

Adding the First Data Source

Now that we've got Grafana up and running, the fun begins. Click the Grafana logo at the top left, then select "Data Sources".

Grafana has native support for InfluxDB, so we don't need to download any special packages to make a connection to the InfluxDB that we created in Part 2. Mapping the data source is super simple, just select "InfluxDB" as the data source type, and give it a name. Provide the URL to the InfluxDB instance, complete with the port (8086). Also change the Access type to "direct" since we don't need a proxy in this example. Lastly, specify the name of the InfluxDB database. I generally use the database name as the name of the data source, but that's personal preference.

Click "Add", and you should be presented with a screen indicating your data source was correctly added and working. This process validates connectivity to InfluxDB, so as long as you see a success message here we're good to go.

Our First Dashboard

There's a ton we can do with Grafana to display the devices on our SmartThings platform, or connect to additional services outside the purview of SmartThings. In future blog posts, I'll expand on some of the more complex capabilities here, but for now we'll just create a dashboard that shows the status of some of our sensors.

I'm a big fan of showing a nice clean and easy to read snapshot of sensors; especially contact sensors and water sensors. If a door has been left open or if a water sensor is wet, I want to easily see that. Grafana is the perfect platform for displaying these types of widgets. For our first dashboard, we'll show some temperatures, door sensors and water sensors.

Plugins

Everything you see on this dashboard is configurable within the UI, except for the installation of the Clocks plugin. Solely for that reason we'll start there. On the Grafana server, simply install the clocks plugin by running sudo grafana-cli plugins install grafana-clock-panel, then restart the Grafana server with sudo service grafana-server restart.

There's also an extensive list of plugins available on the Grafana website.

Creating a Dashboard

Click the Grafana logo at the top left and navigate to "Dashboards" > "New".

We're now taken to a brand new, blank dashboard.

Let's start by adding the clock widgets. The first widget you add to the dashboard is always the easiest since the panel selector is already displayed. Click "Clock" to insert a new clock panel.

Now click on the "Panel Title" to display the panel's options, and click "Edit".

Here you'll see the options on a pretty basic panel.

On the "General" tab, set the title (in this case "UTC TIME"), and set the dimension properties (span of "4", and height of "100px").

On the "Options" tab, we'll set the time format to 24 hours, and set the UTC offsets to 0 so that we get a UTC clock.

Now, simply click the "X" at the right of the options panel to go back to the dashboard.

Panels can easily be duplicated, which is useful if you want to make another one with the same formatting and don't want to have to reconfigure everything by hand. Click the panel title (now "UTC TIME") to expose the panel's options, and click "Duplicate".

Now we'll edit one of the two clocks, so that we can make one local time. To do this, click the panel title > "Edit". On the "General" tab, change the title to "LOCAL TIME". On the "Options" tab, change the UTC offset to match your timezone. In my case -4 Hours for Eastern Standard Time. Click the "X" to the right of the options panel to go back to the dashboard.

Now, let's add some data from our SmartThings hub, starting with the hub's "Mode" (used to indicate Home, Away, etc.). Hover over the top left edge (vertical ellipsis icon) of the row containing the clocks, and click "Add Panel".

Add a "Singlestat" panel, by clicking it from the panel selections.

As with the other panels, click the new empty panel's title to expose the panel options, and click "Edit" to configure the panel.

You'll instantly see that this panel type has a lot more options than the clocks did, starting with this Metrics tab, which allows us to define the query or queries that will surface data for the panel.

The query wizard is incredibly intuitive, and in most cases helps you build your query by restricting the types of measurements and where conditions you can use based on your previous selections. Click "select measurement" and select "stLocation". Next, click the word "value" in the select field condition, and you'll see a list of fields available to be used when querying "_stLocation".

Select "mode", then click "mean()" > "Remove" to remove the default aggregation. Since the value that is reported for "Mode" is going to be text, click the "Plus" after "field(mode)", and navigate to "Selectors" > "Last". Last allows us to query for the last value in the data set for this field, helping to ensure that we get the current value displayed in our dashboard. We'll talk more about different selectors and aggregations in future posts. Lastly, click the "null" inside of "fill(null)", and select "Previous". Using a "Previous" fill will select the last data point in the previous interval if there isn't a data point in this interval. Let's say your mode hasn't changed in 3 hours, but your dashboard is reporting in 10 minute cycles. Without a "Previous" fill, the current reporting interval would have no data.

Finally, simply set your panel title (in my case "SMARTTHINGS MODE"), and close the options panel to go back to the dashboard.

Panels can be reordered within their row by simply clicking and dragging the panel title to alter the panel's position.

Now let's add a new row, by clicking "Add Row", and select a "Singlestat" as the first panel in that row.

What you do with this panel will depend on what sensors you have available, but I'll provide a couple of examples that you can reverse engineer to meet your own needs.

Let's start with a temperature sensor (in this case the Fibaro Motion Sensor, that also reports luminescence, temperature, and shock). By using a "WHERE deviceName equals" condition, we can ensure that we're pulling the data from a specific device. In this case "FROM autogen temperature WHERE deviceName = Garage Motion Detector". We want to select the field(value), and like before, ensure that we're looking at the last value in the data set by using the last() selector, and fill(previous).

Next we'll move to the "Options" tab, and make our panel a little easier to read by adding some color coding. We'll set the stat to "Current" since it's the most recent and/or current value for the sensor, and set the Unit to "Temperature" > "Farehneit (°F)".

In the coloring section, we'll check "Background" to color the box behind the value, and define some thresholds so that the color changes dynamically based on the value. In this case thresholds of 32, 90, 100 ensure that the box is blue for any value below freezing, green for any value between 32 and 90, and red for any value between 90 and 110.

We'll also enable Spark lines, so that the panel gives you a quick glance of the temperature trend of that sensor.

Finally, we'll close the options panel to go back to the dashboard.

Up until this point, we've simply been displaying the value that comes right out of the sensor. That doesn't really work for us if we want to display the status of a door sensor. These sensors report a binary value to the controller, and you probably don't want to see "0" or "1" on your dashboard. Using Grafana we can translate numerical values to text, and display our own user-friendly values for our sensors.

To display this, we'll add a new singlestat panel to show the Open or Closed status of a garage door. Much like our previous queries, we'll use the last() selector and fill(previous) to get the most recent value, but this time we'll use "valueBinary" for the field, instead of "value".

The sensor I'm using reports 1 when the sensor is closed, and 0 when open. We'll set the coloring on the Options tab to fill the background with Red if the value is less than 1, and Green if it's 1 or more. This means that our panel will light up Red when the garage door is open.

Lastly, we want to make our panel say something a little more meaningful than "1" or "0". Using the "Value Mappings" tab, we'll set "value to text" mapping, and define values so that "1" translates to "CLOSED" and "0" translates to "OPEN".

And once again, close the options panel to return to the dashboard and admire what we've learned.

My Aeotec Water Sensors (located under each furnace to detect surplus condensate runoff) are configured in exactly the same way. Simply duplicate the "Garage Door" contact sensor panel, change the device names to poll the right device, and change the value mappings so that "0" translates to "Wet" and "1" translates to "Dry". I also had to reverse the thresholds to indicate 1,0 instead of 0,1 used for the contact sensor, since the water sensors report 1 when Dry.

The final trick to learn, is that any panel in Grafana can be edited with JSON. Click a panel's title, then click the "hamburger" icon, and select "Panel JSON".

You're now shown the full JSON object that makes up a given panel's configuration. This allows you to set precise colors, or make changes to the panel configuration that aren't permitted within the UI. This also lets you easily copy panels from one dashboard to another by simply copying and pasting the JSON.

A full list of all the panel JSONs displayed in this post are available at github.com/webdes03/Grafana.

Dashboard Refresh

The last step is to define the refresh interval for your dashboard by clicking on the "stopwatch" symbol at the top right corner of the dashboard.

By using now-24h or similar expressions in the "From" time range, the dashboard will automatically update the data based on the current time. Also be sure to set a refresh interval to ensure the data is re-queried and the metrics displayed on the dashboard are current. Lastly, save your dashboard by using the Ctrl + S shortcut, or by clicking on the save icon in the header.

Kiosk Mode

Grafana supports a kiosk mode right out of the box, which is great if you want to display your dashboard on a computer monitor or TV. Simply append "&kiosk=true" to the end of your dashboard's URL, and any unnecessary UI component is hidden for a nice clean look.

Next Steps

We've now got Grafana up and running, it's reporting on values from our SmartThings devices, and that data is being sent reliably. In future Home Automation posts, we'll build more advanced visuals in Grafana, and use the principles of SmartApps from Part 3 to deploy more custom components to our SmartThings ecosystem.

Trending & Analyzing SmartThings Devices (Part 3 of 4)

Table of Contents

Publishing Custom SmartApps

To deploy a custom SmartApp, you first need to login to SmartThings Graph. Be sure to login with the same account that you use to login to the SmartThings app on your phone. I originally setup my SmartThings hub using a separate account than the individual user accounts for the app. This caused issues when publishing apps, since the user that publishes the custom SmartApp via Graph is the only user that can install the SmartApp using the iOS or Android app. It sounds, confusing (and it can be), but we'll walk through it all here.

Once you've logged in, navigate to "My SmartApps" on the top navigation.

There's a number of ways to add SmartApps to SmartThings. The easiest is to copy and paste the code of a SmartApp as a new app. In a future post, we'll actually connect SmartThings to GitHub so that we can publish our own apps in a more robust way, but we'll follow a crawl-walk-run mentality here and introduce these concepts slowly.

Click the "New Smart App" button at the top right, then click the "From Code" tab. You'll be presented with a screen like the following.

Into the white code box, paste the following script, then click the "Create" button.

Credit where credit is due. This SmartApp was originally created by David Lomas and is also available at his github repo. I've posted it on my own repo due to the requirement that the script be configured for your specific SmartThings groups, which I'll explain below.

Once you've clicked "Create", you should see a message that the "InfluxDB Logger" SmartApp was created.

This SmartApp is a little special, because we want to be able to report what SmartThings Groups our devices are a member of when we send the data to InfluxDB. This allows us to slice and dice our data using the Group, so if we had 10 outlets in one room, and those outlets were all part of a group, we could trend the power usage for that room.

Unfortunately, a limitation exists in SmartThings today that prevents a SmartApp from querying the group names. This means we need to manually set the group names in our SmartApp, and update them any time we create a new group.

From the navigation at the top of the screen, click "My Locations", then click the name of your location. If you scroll down, you should see a section called "Groups" that contains a list of your groups.

For each of these groups, click the group and make note of the group name, and the group ID. The ID can be found by looking at the URL of the group page. As an example, my Office group has an ID of 40cf3ca5-a8c0-48d7-aa29-934e8fe46e21.

Once you've made a list of all your groups and their IDs, click "My SmartApps" again to go back to the SmartApps list.

Click on the InfluxDB Logger SmartApp, and scroll all the way to the bottom of the code until you see the group listing, and add each of your groups so they look like the following.

Finally, click "Save", and then "Publish" -> "For Me". Verify that you see a "SmartApp published successfully" message.

Installing Custom SmartApps

Now that the custom SmartApp has been published, we can install and configure an instance of it. The act of publishing simply makes the SmartApp available for use within your SmartThings environment.

From your SmartThings app, navigate to "Automation", select "SmartApps", and click "Add a SmartApp".

At the bottom of the list, tap "My Apps".

You'll now see the "InfluxDB Logger" SmartApp that we just published. Tap it, and we'll install an instance of this app and configure it.

Enter the static IP for your InfluxDB server in the Host field, and leave the port set to 8086. In the Database Name field, enter the name of the database we created, SmartThings. Since we didn't setup authentication to InfluxDB, we don't need a username or password, so leave those blank.

The Soft-Pooling Interval allows you to define a period of time (in minutes) that the InfluxDB Logger SmartApp will poll all of the selected devices for their status/values and send that data to InfluxDB. This soft-poll happens in addition to state changes, which are automatically sent to InfluxDB.

Lastly, we need to select the devices that we want to monitor and report to InfluxDB. Select all of the devices you want to monitor, then scroll down and enter a name.

When you're finished with your configuration, tap "Done", and you should now see your new InfluxDB Data Monitoring app on your SmartApps list. This SmartApp is now running and sending data to InfluxDB for the devices you selected.

There's a number of considerations for selecting devices to report status on. Due to the fact that you can install multiple instances of a SmartApp, it's possible to create another instance that contains different devices with a different soft polling interval. This is useful if you want to see more granular data for different types of devices. As an example, maybe you want your energy meter devices to report their power consumption every minute, but you only want thermostat temperatures to be reported every 15 minutes.

Validating Data Connectivity

Now that we've published the SmartThings SmartApp, installed an instance of the SmartApp, and connected it to InfluxDB, we can now confirm that InfluxDB is receiving data from the SmartApp.

After going back to the InfluxDB server and running influx to enter the InfluxDB Shell, we can now run USE SmartThings to connect to our SmartThings database, and then run a simple query like SELECT * FROM water to display any water sensors that we selected for monitoring. Obviously what you check for here will vary based on the devices you selected. If we see data returned, we know that everything is working.

Next Steps

We've now deployed a custom SmartApp, configured it for our groups, selected the devices we want to monitor, and connected it to our InfluxDB server. In part 4 we'll install Grafana and create our first dashboard to help analyze the data we're collecting.

Trending & Analyzing SmartThings Devices (Part 2 of 4)

Table of Contents

What is InfluxDB

If you're super interested in all the little details, you should absolutely go and check out the InfluxDB website, but in a nutshell, InfluxDB is a time series based data platform. Unlike other database platforms that store data based on a primary key that you define, InfluxDB uses a timestamp as the primary key for data points; I'm obviously over-simplifying this to make it a little easier to understand. InfluxDB supports the auto-expiration of data—think of this as a TTL on a DNS entry if that helps you—which automatically purges the data at the end of expiration. The other big advantage to InfluxDB is that any past experience you may have with say SQL, is applicable here. The query language and syntax of InfluxDB has a lot of similarities with SQL, and you'll see that when we get to Part 4 and build our first dashboard.

Getting Started

Before getting started, make sure your OS is up to date. I'm using Ubuntu 16.04, so running a sudo apt-get update && sudo apt-get upgrade && sudo reboot does the trick. If you're using a local device (such as a virtual machine, Raspberry Pi or Pine64) you're also going to want to make sure you have a static IP set.

Installing InfluxDB

InfluxDB stores all of your data in UTC (coordinated universal time). To make things a little easier on ourselves, we'll first set our server to also use UTC.

sudo timedatectl set-timezone Etc/UTC

Next we want to download the install package for InfluxDB. The InfluxDB website contains a list of the packages available for different operating systems. Simply pick the one that matches your OS and run the commands. In my case for Ubuntu 16.04:

wget https://dl.influxdata.com/influxdb/releases/influxdb_1.3.2_amd64.deb

sudo dpkg -i influxdb_1.3.2_amd64.deb

Now it's just a matter of starting InfluxDB:

sudo service influxdb start

Configuring InfluxDB

Now that InfluxDB is installed, we just need to do a little configuration before we can start sending data to it. The first step is to connect to the InfluxDB shell by running influx.

Now we need to create a database that we'll send the data to from SmartThings by running CREATE DATABASE SmartThings, and validate that the database was created by running SHOW DATABASES.

A Note About Retention Policies

When you create a new database, like we did with SmartThings above, InfluxDB automatically creates a default retention policy named autogen that has infinite retention. we can validate this by running SHOW RETENTION POLICIES ON "SmartThings".

You can read up on how to manage retention policies in the InfluxDB documentation, but for our purposes we're just going to modify the default retention policy to hold our data for 55 weeks with a shard group of 1 week. This means that we'll capture a rolling 55 weeks of data, and drop 1 week of data when the data is purged. I chose 55 weeks instead of 52 purely so I have the ability to look back at a calendar year's worth of data for up to 3 weeks. So if I happen to not look at January 1 - December 31 on exactly the 31st of December it's no big deal. Having an extra 3 weeks simply buys me a little extra time.

Authentication

Note that we didn't setup any authentication as part of this install of InfluxDB. This means that InfluxDB will accept any request that it is sent regardless of where that request came from. That's fine for be because I'm using a local virtual machine inside my own network's firewall. If you've chosen to host your InfluxDB on a publicly accessible server, such as at Digital Ocean, you'll absolutely want to harden your server and configure InfluxDB authentication.

Next Steps

InfluxDB is now up and running on our server, we've created a database (SmartThings), set a retention policy to keep data in that database for 55 weeks, and in part 3 we'll start sending data to InfluxDB from SmartThings.

Trending & Analyzing SmartThings Devices (Part 1 of 4)

Table of Contents

Introduction

Back in the Home Automation Kickoff post, I talked about the power of the mFi Controller to record, trend, and analyze device and energy usage. One of the huge disappointments with Ubiquiti ending development of the mFi Controller was the end of development related to these capabilities. SmartThings has proven itself to be a fantastic platform for the integration and management of physical devices, but offers essentially no analytics around those devices. You can pick up devices like the fantastic Aeotec Smart Switch, which will report power measurements to the SmartThings UI, but that data is simply a reading at that moment. I feel compelled to point out that I have said previously that the Ubiquifi mFi mPower Mini is the best WiFi controlled switch I've ever used. If that device had a cool Z-Wave cousin, it would in fact be the Aeotec Smart Switch.

SmartThings has no mechanism today to do anything with the data its' devices report, other than use it as conditions in automation (which in itself is incredibly powerful—I'm in no way trying to belittle those capabilities). Nor do I believe SmartThings should try and become an analytical platform. They've cracked what they're good at, and trying to be everything to everyone is never a good product strategy (spoken as a product guy).

But all that said, if you're using devices like the Aeotec Smart Switch, you probably want to be able to trend that data. This is a perfect scenario for an external platform to store, analyze and trend this data, and a perfect example of a more powerful, custom SmartApp for SmartThings. In the next couple posts, we'll deploy and configure InfluxDB to record the data from SmartThings devices, a custom SmartThings SmartApp to send the data from SmartThings to InfluxDB, and we'll use Grafana to visualize and trend the data from InfluxDB.

These four posts will be the foundation for many of the future posts in this Home Automation series, and I've broken the content into separate posts to make them a little easier to digest.

Platform Concepts

For this project, I've created two Virtual Machines (one for InfluxDB and one for Grafana). I've taken this approach for a couple of reasons. The first being that I plan on having a lot of devices and a lot of data to store (data that I'd like to keep for a long time for trending). The second (and perhaps more relevant) reason is that I have hardware here that I can run these Virtual Machines on. Depending on your scenario there's a few alternative approaches you can take:

Raspberry Pi

I'm a huge fan of the Raspberry Pi. It was the first affordable, small form factor, single board computer that really went mainstream. You can find them pretty much anywhere, and there's a huge amount of community support. If you opt to go the Raspberry Pi route, I'd strongly recommend a Raspberry Pi 3, since the third generation has a bit more horsepower. You'll also want to make sure you get a good, fast and large enough Micro SD to hold the software and the amount of data you want to keep. Something like the Samsung Evo Plus cards is usually a good choice here.

Pine64

You may have seen my previous posts on the Pine64, but if you haven't I'll summarize here. Like the Raspberry Pi, the Pine64 is a single board computer, with a few subtle differences. The quad-core ARM processor is faster than a Raspberry Pi 3, and the Pine64 is available in a 2Gb variant that comes with 2Gb of memory as opposed to the Raspberry Pi 3's 1Gb. In many ways, the Pine64 is a lesser-known, but more powerful AND cheaper option. It's newer, so OS compatibility is a little more spotty, and you have to be willing to do a little more work to get up and running. If that's your thing, you can follow my previous posts to help you get started as the steps will be very similar here.

Cloud Host

If you'd rather not mess with hardware, and just want to get started, spin up a cloud virtual machine at Digital Ocean. Digital Ocean is a great platform for these sorts of projects. Their servers (called Droplets) are as little as $5/month, come with SSD storage, run just about any flavor of Linux you could want. Oh, and they provision in as little as 60 seconds.

Introduction to SmartThings SmartApps

SmartApps are written in Groovy, and can be found within the "Add a SmartApp" menu inside of SmartThings. SmartThings provides a number of SmartApps and SmartApp categories right out of the box, so you can just pick an existing one and be up and running. In addition to that, you can self-publish your own SmartApps directly within SmartThings Graph, or publish SmartApps to GitHub for yourself or others to use. I'll talk more about the latter options when we get to creating our own SmartApp in a future post. You can think of a SmartApp almost as a Routine on steroids. SmartApps are a way to run complex logic, calculations, or even call external services as part of the automation.

To illustrate the power of the out of the box SmartApps, let's look at the Vacation Lighting Director SmartApp, which automatically makes your home appear occupied when you're away. This is the modern alternative to running around the house setting individual light timers before you hit the road for your trip.

SmartApps are located right alongside "Routines" in the automation panel, and clicking "Add a SmartApp" will present you with a list of available/installed SmartApps.

Scroll down and tap on the "Lights & Switches" category, and you'll find a list of lighting related SmartApps. The list of apps that SmartThings includes out of the box varies from time to time, and you'll notice that some of the SmartApps, including the "Vacation Lighting Director" were not actually written by SmartThings. If you read my Home Automation Kickoff post you'll remember my praise to SmartThings for embracing the community and extensibility of the SmartThings platform. The fact that they're actually evaluating, selecting, and publishing best of breed apps under names other than their own is one big example of this.

If we select the "Vacation Lighting Director" you'll see that unlike Routines, SmartApps may actually have many layers of configuration options to them. While this is an example of a powerful SmartApp, they really can be taken as far as you (or another developer) want to take them. Here we're prompted to do some setup, configure additional settings, and name our app.

Within the setup menu, we can set the mode(s) that the app will run during. Modes are a great way to isolate sensor logic or automation when it is dependent on different conditions. By default SmartThings has a mode for Home, Away, and Night, but custom modes can also be created. As an example, you could use a custom "Vacation" mode that changes alert and notification properties of a SmartApp so that you get notified under different conditions than if your home was simply in "Away" because you went to the grocery store.

The options highlighted in red are options that we have to complete for the SmartApp to function. For now we'll set it to run when "Away", and scroll down for the rest of the options.

The SmartApp needs to know what lights we want it to turn on and off to represent your house being occupied, how often we want it to change those lights, and how many lights should be on at a time. For this example I only selected two lights, but you could conceivably select many, many more.

Tap "Done" to go back to the main SmartApp configuration screen, then tap "Settings" to see the additional options on this SmartApp. None of these options are required, but we have the ability to set a delay on the start. This is useful if you've configured the SmartApp to run under say "Away" mode, and you accidentally put your system into "Away". This gives you a window to set it back before the simulation runs. We also have options to only run on certain days of the week, or if presence indicators indicate specific people are home. The latter is useful if you want the simulation to automatically stop when you return home.

Now that we've completed the configuration of our SmartApp, it's just a matter of clicking "Done" to go back to the main Automation screen. The SmartApp has been configured and will now run automatically when the configured conditions are met.

Getting Started with SmartThings - Sunrise & Sunset Rules

Back in the Home Automation Kickoff post, I mentioned my past frustrations getting simple things like sunrise and sunset rules to work. For this example, we'll be using a simple SmartThings routine to turn on an outdoor GE Z-Wave module at sunset.

To illustrate how simple this is, first go to the Automation tab of the app, make sure that you're on the "Routines" tab (we'll talk more about SmartApps later), and tap "Add a Routine". We'll name this routine "Outdoor Lights on at Sunset".

You'll see that routines are broken down into super intuitive categories, like "What do you want to happen?". Tap "Turn on these lights or switches" since we want to turn On some lights at sunset, and select the appropriate devices to turn on.

Scroll down and tap the option to "Automatically perform 'Outdoor Lights on at Sunset'" so that we can define when we want this routine to run.

You'll see that you can now select from a number of triggers to start your routine. For this example, tap "At sunrise or sunset".

If you wanted to turn on your lights 30 minutes before sunset as an example, you can simply configure it. Make sure you select whether this is for sunrise or sunset, and enter any time offset you wish.

Once you tap "Done", you'll get an alert saying your routine is running, but you're taken back to the triggers screen, not back to the main routines screen. One of the things that's really slick about the SmartThings interface is the ability to do complex rules without a lot of thought. Since we were taken back to the triggers, we can now easily add a second (or third, or forth) trigger to our routine. This is particularly useful if you want to use multiple sensor inputs to control a device. You'll notice that there's a blue bar next to any trigger that is enabled for this routine.

Once you hit "Done", you're now taken back to the main routines screen, and you'll see that your new routine is ready to go. It's already running in the background, so 30 minutes before sunset, it'll automatically run, or you can run it manually just by clicking on it.

Home Automation Kickoff

I've long planned to start a Home Automation series in an attempt to document the deployment, configuration and monitoring of various home automation components. This journey really started over a year ago when my wife and I built our house. With network drops run to anywhere that I thought we might ever want to put anything, I saw home automation as a valid excuse to apply my passion for technology outside of work and get back to some of my more "maker" roots.

I really had a few goals that I was trying to accomplish. Energy conservation and trending were at the top of that list. I've always assumed that solar panels were in our future at some point, but without knowing what our true energy usage was, or where we were wasting energy were the first two problems to solve before considering solar in order to ensure we could right-size a solar deployment and cover a large percentage of our average energy use.

Secondary to the energy monitoring was the security and comfort side of home automation. Lighting presets, more granular control over HVAC, sunrise/sunset rules, presence driven rules, etc. were all things I was excited to learn more about and develop over time.

Home Automation is a pretty exciting field. The development of new technology and protocols, and the consumerization of those capabilities has been happening at a pretty quick pace. In the section below, you'll see that I was maybe a year early of the market in terms of what I really wanted. This series will share the successes (and failures) I've had, and hopefully prevent you from making some of the same mistakes that I made early on.

In the beginning...

Things started off pretty smooth. I bought a bunch of Ubiquiti mFi equipment (switches, dimmers, sensor controllers, power strips, etc.) and setup about a third of the house, starting with my home office. mFi provided a fantastic web-based controller that allowed you to trend energy consumption. Furthermore, it allowed you to define your cost per kW from your energy company, and it would help you to understand how much of monthly energy bill should be contributed to individual devices.

Furthermore, the mFi controller provided the tools necessary to create advanced rules and define pretty complex sensor logic. I say all of this in past tense, because just a few months after getting everything up and running, Ubiquiti announced they were ending support for mFi. While they still sell the hardware for folks that want to manage it themselves, the controller itself no longer being developed or supported. I still hold the opinion that the Ubiquifi mFi mPower Mini is the best wifi controlled switch you can buy. I'm still deploying those switches (even without an mFi controller), most recently to control my 3D printer.

Where to go next...

Facing the dilemma to either continue buying mFi hardware and deal with an end-of-life controller, or to completely start over was a daunting prospect. mFi was all wifi based, which had it's own advantages, but no more updates for the controller also meant that the mFi platform itself was never going to support non-mFi devices. This sort of killed my dream of one unified platform to control the entire house. The wishlist of feature requests from customers was never going to be handled by Ubiquiti, and the prospect that they may randomly decide to also stop selling the hardware at any time was equally as worrying. I decided it was best to try a few alternatives out before making a final decision. It's important to note that the comparison of OpenHAB, Wink and SmartThings was all done 6-9 months ago, so my views of these products depicted below may not accurately represent advancements made in those platforms between then and now. I share these experiences merely for context and to tell the story of what led me to where I am today.

OpenHAB

The first alternative I tried was OpenHAB, running on a Raspberry Pi 2 with an Aeotec Z-Stick for Z-Wave control. I desperately wanted to love OpenHAB. It was open source, extendable, and could conceivably do whatever I wanted (if I was willing to put the time into getting it to all work).

I'll preface this with the statement that this was the late OpenHAB 1 days, before the release of OpenHAB 2. This was an awkward time because the focus of the developers was understandably OpenHAB 2, but OpenHAB 2 was not really ready to be deployed. I'm pretty technical, but as much as I tried to get OpenHAB up and running in a stable and reliable way, it just wasn't happening. I also quickly learned that the home grown nature of OpenHAB also meant that it was pretty lacking in consumer support. This meant there was no great iOS app and no great user interface, resulting in something that you really couldn't expect the other members of your household to happily use. Pairing new devices was a chore, and things frequently just didn't work. The big draw to OpenHAB was that it was open source, meaning that when it didn't work you could crack it open and debug Java on your own, but even I didn't want to do that just to turn my kitchen pendant lights on.

Wink

Looking for a more commercially viable option, I picked up a Wink Hub. As with OpenHAB this was an equally awkward time for Wink. The Wink Hub 2 was known to be in development, but the company itself wasn't tight lipped on what it would support, or if it could close any of the gaps in the original Wink Hub. I'm not positive if it's still the case, but at that time, Wink only supported devices that were formally marked as "Compatible with Wink Hub". This meant that finding switches, outlets, sensors, etc. was incredibly difficult. You couldn't just grab any Z-Wave switch and pair it.

One of my big draws to Wink was the very prominent promotion of the fact that there was an API. I took this to mean I could close any gaps in the product on my own, by coding against the API. At the end of the day, that API was pretty weak. I found myself struggling to even turn on an outside light at sunset. I returned the Wink and moved on.

SmartThings

Still on the hunt for a commercial product that could handle advanced rules and a wide array of devices, I next picked up a SmartThings Smart Home Hub. The SmartThings hub supports a variety of protocols, and [probably] thanks to the Samsung ownership has pretty widely adopted device support. It seems like almost any mainstream Z-Wave or Zigbee device you pick up has a "SmartThings" logo on the packaging. So far, so good.

The next big surprise for me was when I discovered SmartThings Graph. Not only does SmartThings offer a feature-rich API, they've built an entire UI around API management. Using SmartThings Graph you can create your own device profiles to add support for custom sensors or devices that don't have a formal SmartThings device profile. You can also create your own smart apps to control devices or handle more advanced sensor logic. And probably the best capability, is that those custom apps and devices you create are all surfaced in the incredibly user-friendly SmartThings iOS and Android apps.

I suddenly found myself with a platform that was a consumer product, supported by the manufacturer and a huge user community, that still provided the control and extensibility that a power user like myself wanted. SmartThings seem to have hit that balance between an open source project, consumer product, and hardware ecosystem that I didn't really think was going to exist. On top of that, they seem to develop at a pretty good rate with frequent firmware updates for hardware and app updates for iOS and Android.

All of these points led me to select SmartThings as my home automation platform going forward. Stay tuned to the Home Automation category. In future blog posts I'll share the setup, configuration, monitoring, and challenges as I continue on this mission to fully automate our home.