Gadgetizing BioHeatMap: Colorful Squares of Energy

Shifting gears to coding a Google Gadget is a very interesting and different experience than previous endeavors. My past projects such as Greenometer and WattDepot-Applications have yielded fruitful results, but creating a Google Gadget is something that can be immediately distributed and displayed directly on the web without any complication between servers or multiple frameworks.

With my partner Kendyll, and using his idea of implementing a BioHeatMap visualization, we decided to take on this new challenge as one step further in creating a tool student's can use for the UH Energy Dorm Competition.

An example of a BioHeatMap visualization looks like this:

It essentially takes in a matrix of data and conditionally formats the data by displaying various colors. It finds the max and min values, assigns two colors to them and all other values in-between, assigns color appropriate to how close those values are to the max and min. We are currently exploring two options:

1. Rows are the floors. There are 24 columns, one for each hour of the day. The cell is the *cumulative* energy consumed during the competition for that hour of the day. That gives people on the floor a sense for when they're using a lot, as well as how they compare to others.
2. Rows are the floors. There are 28 columns, one for each day in the competition. The cell is the total energy consumed by each floor on that day. (A cell is blank if the day is in the future so there's no data for it yet.) This shows which floors used a lot on a particular day. If a floor is mostly red, it's using a lot of energy, and if a floor is mostly green, it's using very little. It also highlights which floors are consistent vs. which floors are "spiky"---one day using a lot of energy, the next day using very little.

Since this is something relatively new to take on, the first thing on the agenda was to get a working "dummy" version of the BioHeatMap visualization as a Gadget. By looking at the BioHeatMap visualization page, it was a little discouraging to see that most of the development for the BioHeatMap has ceased since the latter part of last year. But by looking at other visualizations that are backed by Google, I came across the Google Visualization Gadget Gallery. Sadly the BioHeatMap was not listed under here since it was originally created by Institute for System Biology, but I was still lucky enough to dissect one of the listed gadgets, mainly Gauge. I chose the Gauge since another group is going to try and implement it for the competition, so I helped myself to see what they would be dealing with.

The actual .xml file is being hosted in the Project Hosting here.

But for those who want to try out the "dummy" BioHeatMap the link to add to your iGoogle page is: http://wattdepot-ui-googlegadgets.googlecode.com/svn/trunk/BioHeatMap/test-bio-heat-map.xml

The data source is just a Google Spreadsheet: http://spreadsheets.google.com/ccc?key=0AhWy1L2qDnFYdGtuQTlJME9rN2pDOFhGRFBuMHZsbFE&hl=en.

Right now only I and Kendyll have access to edit the Spreadsheet, so if you want access, just send me an e-mail and I'll add you to the Google Doc.

Some of the issues I'm currently at is the default height of the Gadget is extremely long. I will have to correct that this week and try and get it to fit "snug" with the visualization. A general issue I came across is while browsing the example pages for the BioHeatMap. Most of the examples look pretty cool, however this one caught my eye:

It looks like a total mess, and judging by the size, I'd say that's a typical size we'll be looking at. Although I'm not sure how sporadic the data has to be in order to get it like that, but just by first glance it's hard to interpret or make heads/tails.

Advanced Visualizations, New Frontier on Familiar Grounds

As the planning of the UH Dorm Energy Competition continues, there are still other things we are considering to implement. Shifting from WattDepot Applications, I am going to be working on a new topic, Advanced Visualizations. This new group is sort of like the WattDepot-App's Visualizer, except we are going to explore the different Google Visualizations and see how we can apply some of them to WattDepot.

The main group consists of , but I will be working with Kendyll Doi, the same Kendyll Doi from WattDepot Applications, as a sub group.

In order to get started, Prof. Johnson provided a few suggestions of what we could do. Right now, I'm just looking at the different suggestions and getting the feel of how I would implement them. Details about which framework we should use, either Wicket or Google Gadget, will be sorted out sometime this week.

Here are some of the suggestions that I think would be duo-able:

Rolling Update of Sensor Data

This visualization basically combines the idea of the monitor (i.e. get the latest value) with the trend line (show changes over time).

The idea is to have a visualization where you specify the source(s),the update interval (i.e. 30 seconds, 1 minute, 1 hour, 1 day), and the window size (i.e. 5 minutes, 1 hour, 1 day, 1 week), and the result is a chart that is refreshed every update interval and shows the last window size of data.

The reason why I like this idea is that in theory it sounds like the WattDepot Visualizer, but with an added refresh event and without the messy-ness of multiple sources, although in theory it could accommodate multiples (since it already does that). The only problem I'm seeing right now is that if the refresh interval is set too short, say 30 seconds.

---

Gauge

An alternative visualization for the "monitor" gadget. Also has red/yellow/green capability like the stoplight.

A simple enough visualization, similar to a speedometer in a car. However, the challenge with this one is that the example Gauge visualization shows that the limits between red, yellow, and minor are hard-coded values out of 100. My thoughts would be to try and scale all the values to 100, but that would require some type of maximum calculation for all data being presented, and since the data would be streaming live, it would be hard to set a number to scale to 100.

Indeed there are a lot more other Visualizations to choose from, but at this point the refreshing annotated timeline, and gauge seem to be the most plausible. The other visualizations such as Bio Heat Map for displaying energy generated and Term Cloud for quickly displaying a source that's consuming the most energy also seem plausible, but Kendyll and I are going to play around with the refreshable timeline, and possible Gauge to get us going. Our first weekly meeting for this new group with Prof. Johnson is scheduled for this Thursday. More details to come as Kendyll and I decide which framework and visualization to attempt to implement.

Rolling Out Milestone 2: WattDepot-Apps v2.0

Introducing WattDepot Applications version 2.0! The second milestone is here, and with it comes a newly completed application, the WattDepot Monitor, and a whole new look to the WattDepot Visualizer. The past month has been devoted to the improvement of the Visualizer, as well as the implementation of the Monitor. Sadly, the WattDepot Browser is going to have to wait for another developer to pick up as the Visualizer and Monitor took up most of our resources.

Our updated Google Hosting site for WattDepot-Apps can be found here, complete with featured wiki pages for end-users as well as for developers curious about WattDepot and our project. For this latest release, there are 3 new downloads which can be found on the Downloads page. They include the individual .jar files for the Visualizer and Monitor, and the distribution .zip file which contains all the source code for each application.

I will briefly go over each application, and some of the new features. For more information you can look at the various wiki pages here.

WattDepot Visualizer

---

The WattDepot Visualizer is an application that uses Google Visualization's Annotated Timeline and Table to show data about a single or multiple sources throughout a given beginning and ending date in a WattDepot server.

Here's a screenshot of what the Visualizer looked like for milestone 1:

The major issue the Visualizer had in milestone 1 was that it could only display data about a single source, hence the drop-down menu. But for milestone 2, we were able to get that working properly, and now the Visualizer looks like this:

There was some discussion about the overall layout of the components. In milestone 1, the Display Data and Source started to hang on the left side, but ultimately we decided it would look more efficient if we had all the components at the top going left to right.

When running the Visualizer .jar you can specify three arguments, which are optional. These are the port number which the Visualizer listens through, and the Data and Rest URIs. The Data URI is used to query WattDepot when creating a Google Visualization, while the Rest URI contains sensor data and all information about each source in the server.

One of the tasks I had to do to make the configurable Data URI possible was pass a variable from Java to JavaScript using Wicket constructs. To get started, first I visited the Wicket Mailing List and tried to see if anyone already tried to do what I want. I searched a couple pages, but couldn't find exactly what I needed. There were some variants, but nothing directly I could use. However, searching through Google, I stumbled across a Chillenious' WordPress which was exactly what I was looking for. The post itself was done in '06 so some of the coding needed to be altered, mainly parameterization. What I needed to accomplish a "variable substitution" was to wrap a HasMap around a Wicket component (IModel).

Here's a snippet of the parameterized version:

Here's a snippet of the JavaScript of the variable being substituted:

Overall, the Visualizer works as intended, it may not be the most aesthetically pleasing as it is just a Developer level application, the functionally and groundwork has been implemented and I like how it turned out.

WattDepot Monitor

---

The WattDepot Monitor is an application that displays the most recent SensorData of a given source. The user simply selects a Source, Interval, and Data Type, and the Monitor will refresh every interval showing the most recent value of the selected data type. The development of this application was solely Wicket, which was an interesting experience. I had to learn by example from sifting through the Mail List and also going through the Wicket API.

During the initial phase of the Monitor Prof. Johnson drew a mock-up of how the Monitor should look like:

For milestone 2, here's what the Monitor looks like:

Much of the design of the Monitor was pretty straight forward. One of the new features that we added was a Portable URLs. What it does is pass parameters through the URL and configure the UI components accordingly. Ideally we wanted to have this feature in both the Visualizer and Monitor, but we only had time to implement it for the Monitor. It makes more sense to have it in Visualizer than the Monitor, but we wanted to start on a more simple case, and maybe the next developer can implement the rest. The components in the Monitor are all Wicket while the Visualizer is Wicket and JavaScript so it may require some modifications to work properly.

Closing Thoughts

---

It's been a long and arduous past couple months, yet I've learned quite a lot. A big part of it was just learning Wicket constructs, but the practices we used through Hudson, SVN, and Quality Assurance tools, helped carry the project forward as each step we took to create the final product. The weekly meetings helped set what we needed to do and the constant feedback at each development phase ultimately made the finishing touches on both applications.

Monitoring a Green Future, Tweaking WattDepot Monitor

The second milestone is almost near and by next week Tuesday will have reached the deadline. There has been much progress throughout both milestones for each of the sub-projects. Since last week, the basic functionality of the Monitor was complete. Our weekly meeting with Prof. Johnson and Robert Brewer awaited input on the Monitor as a whole. Much of the comments were on the design itself.

Here's a screen shot of how the Monitor looked last week:

The main comment about the Monitor was that it seemed too loose, there needed something to contain it since it looked like everything was just floating on the page.

After taking into consideration of the comments, here's what the monitor current looks like zoomed in:

Looking back at this past week, I realized what I was doing was more on the basic aesthetics of the Monitor. I was designing something for a kiosk, or for continuous display. So along with the added borders, I changed the font to Sans Serif so it would be better viewed from a distance, and also bold the labels on the right side.

During the meeting, I was on UH's wireless network, which doesn't always have the most reliable connection. So at times, the Internet would cut out, and to my dismay, here's a few screen shots that shocked me:

(Above: No connection on initial load)


(Above: Connection lose during monitoring)

I did not take into account a bad connection, thus some of the error messages actually blended in with the form itself as in the first picture, or a very unhelpful user-end error message as in the second picture. After some adjustments, the Monitor now displays these when the connection fails:

(Above: Drop-down menu just says No Sources when there is no connection on initial page load)


(Above: Right side simply says Connection Failure, and will resume monitoring when connection is restored)

With this new layout in hand, it awaits input for tomorrow's meeting with Johnson. What I am also looking into is a Reset button. For the most part, when the Monitor button is clicked, it simply sends a new form and theoretically should update the Monitor, however, at times when the Java is waiting for a response back from WattDepot, Java becomes dead-locked until the call completes. It is during this time that if a new form is submit, depending on how long the call to WattDepot takes to compelte, the form might revert back to it's previous state and dismiss the new form that was trying to be submit. A Reset button would be a sure fire way to ensure that the form will always submit and update.

We are also looking into creating portable/pretty URLs that take in parameters through the URL and adjust the components to match. So instead of clicking through drop-down menus, a user would simply edit a URL, and it would accomplish the same task. Right now, the page is able to parse arguments through the URL, but ultimately you still have to click "Monitor" button to start the process.

Also, when you click "Monitor" the URL of the active page becomes:

But as a release we want it to generate the appropriate URL in the address bar, not the console where we currently have it:


For this week, what we are aiming to accomplish is both the Visualizer and Monitor applications. We need to have completed the Visualizer and Monitor to the best of our abilities, get a distribution up on our Google Project site, and create new Wiki pages to match. If we're never going to touch WattDepot-Apps again, atleast we will have all the documentation necessary for any developer to pick up where we left off.

It's funny how these little things catch you when you're only programming for functionality and not actual use. The gap between a developer and end-user sometimes gets lost when just programming for a grade and not for actual deployment, which is exactly what this class aims to do.