Advanced Visualizations, New Frontier on Familiar Grounds

As the planning of the UH Dorm Energy Competition continues, there are still other things we are considering to implement. Shifting from WattDepot Applications, I am going to be working on a new topic, Advanced Visualizations. This new group is sort of like the WattDepot-App's Visualizer, except we are going to explore the different Google Visualizations and see how we can apply some of them to WattDepot.

The main group consists of , but I will be working with Kendyll Doi, the same Kendyll Doi from WattDepot Applications, as a sub group.

In order to get started, Prof. Johnson provided a few suggestions of what we could do. Right now, I'm just looking at the different suggestions and getting the feel of how I would implement them. Details about which framework we should use, either Wicket or Google Gadget, will be sorted out sometime this week.

Here are some of the suggestions that I think would be duo-able:

Rolling Update of Sensor Data

This visualization basically combines the idea of the monitor (i.e. get the latest value) with the trend line (show changes over time).

The idea is to have a visualization where you specify the source(s),the update interval (i.e. 30 seconds, 1 minute, 1 hour, 1 day), and the window size (i.e. 5 minutes, 1 hour, 1 day, 1 week), and the result is a chart that is refreshed every update interval and shows the last window size of data.

The reason why I like this idea is that in theory it sounds like the WattDepot Visualizer, but with an added refresh event and without the messy-ness of multiple sources, although in theory it could accommodate multiples (since it already does that). The only problem I'm seeing right now is that if the refresh interval is set too short, say 30 seconds.

---

Gauge

An alternative visualization for the "monitor" gadget. Also has red/yellow/green capability like the stoplight.

A simple enough visualization, similar to a speedometer in a car. However, the challenge with this one is that the example Gauge visualization shows that the limits between red, yellow, and minor are hard-coded values out of 100. My thoughts would be to try and scale all the values to 100, but that would require some type of maximum calculation for all data being presented, and since the data would be streaming live, it would be hard to set a number to scale to 100.

Indeed there are a lot more other Visualizations to choose from, but at this point the refreshing annotated timeline, and gauge seem to be the most plausible. The other visualizations such as Bio Heat Map for displaying energy generated and Term Cloud for quickly displaying a source that's consuming the most energy also seem plausible, but Kendyll and I are going to play around with the refreshable timeline, and possible Gauge to get us going. Our first weekly meeting for this new group with Prof. Johnson is scheduled for this Thursday. More details to come as Kendyll and I decide which framework and visualization to attempt to implement.

Rolling Out Milestone 2: WattDepot-Apps v2.0

Introducing WattDepot Applications version 2.0! The second milestone is here, and with it comes a newly completed application, the WattDepot Monitor, and a whole new look to the WattDepot Visualizer. The past month has been devoted to the improvement of the Visualizer, as well as the implementation of the Monitor. Sadly, the WattDepot Browser is going to have to wait for another developer to pick up as the Visualizer and Monitor took up most of our resources.

Our updated Google Hosting site for WattDepot-Apps can be found here, complete with featured wiki pages for end-users as well as for developers curious about WattDepot and our project. For this latest release, there are 3 new downloads which can be found on the Downloads page. They include the individual .jar files for the Visualizer and Monitor, and the distribution .zip file which contains all the source code for each application.

I will briefly go over each application, and some of the new features. For more information you can look at the various wiki pages here.

WattDepot Visualizer

---

The WattDepot Visualizer is an application that uses Google Visualization's Annotated Timeline and Table to show data about a single or multiple sources throughout a given beginning and ending date in a WattDepot server.

Here's a screenshot of what the Visualizer looked like for milestone 1:

The major issue the Visualizer had in milestone 1 was that it could only display data about a single source, hence the drop-down menu. But for milestone 2, we were able to get that working properly, and now the Visualizer looks like this:

There was some discussion about the overall layout of the components. In milestone 1, the Display Data and Source started to hang on the left side, but ultimately we decided it would look more efficient if we had all the components at the top going left to right.

When running the Visualizer .jar you can specify three arguments, which are optional. These are the port number which the Visualizer listens through, and the Data and Rest URIs. The Data URI is used to query WattDepot when creating a Google Visualization, while the Rest URI contains sensor data and all information about each source in the server.

One of the tasks I had to do to make the configurable Data URI possible was pass a variable from Java to JavaScript using Wicket constructs. To get started, first I visited the Wicket Mailing List and tried to see if anyone already tried to do what I want. I searched a couple pages, but couldn't find exactly what I needed. There were some variants, but nothing directly I could use. However, searching through Google, I stumbled across a Chillenious' WordPress which was exactly what I was looking for. The post itself was done in '06 so some of the coding needed to be altered, mainly parameterization. What I needed to accomplish a "variable substitution" was to wrap a HasMap around a Wicket component (IModel).

Here's a snippet of the parameterized version:

Here's a snippet of the JavaScript of the variable being substituted:

Overall, the Visualizer works as intended, it may not be the most aesthetically pleasing as it is just a Developer level application, the functionally and groundwork has been implemented and I like how it turned out.

WattDepot Monitor

---

The WattDepot Monitor is an application that displays the most recent SensorData of a given source. The user simply selects a Source, Interval, and Data Type, and the Monitor will refresh every interval showing the most recent value of the selected data type. The development of this application was solely Wicket, which was an interesting experience. I had to learn by example from sifting through the Mail List and also going through the Wicket API.

During the initial phase of the Monitor Prof. Johnson drew a mock-up of how the Monitor should look like:

For milestone 2, here's what the Monitor looks like:

Much of the design of the Monitor was pretty straight forward. One of the new features that we added was a Portable URLs. What it does is pass parameters through the URL and configure the UI components accordingly. Ideally we wanted to have this feature in both the Visualizer and Monitor, but we only had time to implement it for the Monitor. It makes more sense to have it in Visualizer than the Monitor, but we wanted to start on a more simple case, and maybe the next developer can implement the rest. The components in the Monitor are all Wicket while the Visualizer is Wicket and JavaScript so it may require some modifications to work properly.

Closing Thoughts

---

It's been a long and arduous past couple months, yet I've learned quite a lot. A big part of it was just learning Wicket constructs, but the practices we used through Hudson, SVN, and Quality Assurance tools, helped carry the project forward as each step we took to create the final product. The weekly meetings helped set what we needed to do and the constant feedback at each development phase ultimately made the finishing touches on both applications.

Monitoring a Green Future, Tweaking WattDepot Monitor

The second milestone is almost near and by next week Tuesday will have reached the deadline. There has been much progress throughout both milestones for each of the sub-projects. Since last week, the basic functionality of the Monitor was complete. Our weekly meeting with Prof. Johnson and Robert Brewer awaited input on the Monitor as a whole. Much of the comments were on the design itself.

Here's a screen shot of how the Monitor looked last week:

The main comment about the Monitor was that it seemed too loose, there needed something to contain it since it looked like everything was just floating on the page.

After taking into consideration of the comments, here's what the monitor current looks like zoomed in:

Looking back at this past week, I realized what I was doing was more on the basic aesthetics of the Monitor. I was designing something for a kiosk, or for continuous display. So along with the added borders, I changed the font to Sans Serif so it would be better viewed from a distance, and also bold the labels on the right side.

During the meeting, I was on UH's wireless network, which doesn't always have the most reliable connection. So at times, the Internet would cut out, and to my dismay, here's a few screen shots that shocked me:

(Above: No connection on initial load)


(Above: Connection lose during monitoring)

I did not take into account a bad connection, thus some of the error messages actually blended in with the form itself as in the first picture, or a very unhelpful user-end error message as in the second picture. After some adjustments, the Monitor now displays these when the connection fails:

(Above: Drop-down menu just says No Sources when there is no connection on initial page load)


(Above: Right side simply says Connection Failure, and will resume monitoring when connection is restored)

With this new layout in hand, it awaits input for tomorrow's meeting with Johnson. What I am also looking into is a Reset button. For the most part, when the Monitor button is clicked, it simply sends a new form and theoretically should update the Monitor, however, at times when the Java is waiting for a response back from WattDepot, Java becomes dead-locked until the call completes. It is during this time that if a new form is submit, depending on how long the call to WattDepot takes to compelte, the form might revert back to it's previous state and dismiss the new form that was trying to be submit. A Reset button would be a sure fire way to ensure that the form will always submit and update.

We are also looking into creating portable/pretty URLs that take in parameters through the URL and adjust the components to match. So instead of clicking through drop-down menus, a user would simply edit a URL, and it would accomplish the same task. Right now, the page is able to parse arguments through the URL, but ultimately you still have to click "Monitor" button to start the process.

Also, when you click "Monitor" the URL of the active page becomes:

But as a release we want it to generate the appropriate URL in the address bar, not the console where we currently have it:


For this week, what we are aiming to accomplish is both the Visualizer and Monitor applications. We need to have completed the Visualizer and Monitor to the best of our abilities, get a distribution up on our Google Project site, and create new Wiki pages to match. If we're never going to touch WattDepot-Apps again, atleast we will have all the documentation necessary for any developer to pick up where we left off.

It's funny how these little things catch you when you're only programming for functionality and not actual use. The gap between a developer and end-user sometimes gets lost when just programming for a grade and not for actual deployment, which is exactly what this class aims to do.

WattDepot Monitor First Iteration

This past week for my Software Engineering class I've been working solely on the WattDepot Monitor. The WattDepot Monitor as a web application that displays the most recent sensor data of a selected source in WattDepot.

While the Visualizer dealt with JavaScript, Java, and Wicket as its background engine, the Monitor was solely based on Wicket constructs. During our weekly meeting with Prof. Johnson, we concluded that the person to do the Monitor would be the one who is most knowledgeable in Wicket. Since I was the one setting up all the other pages WattDepot-Apps uses and other various UI components that the Visualizer uses, I was the most likely candidate for the task. I actually wanted to do the Monitor because some of the elements that the Monitor needed (timer behaviors) were already incorporated in last semesters project, Greenometer.

I've completed the base functionality, with some assistance from my teammate Bao with the time conversion, of the Monitor. Here's a screen shot of the current interface.

Right now the trunk has the most up-to-date version of the Monitor with debug print statements active. As of now, there has not been a .jar created solely for the monitor, but I will not make one until probably after tomorrow's meeting.

The basic How-To goes:

1. Select your source. After you select a source, beneath the drop-down menu a text field auto-updates and notifies you how often the source gets updated.
2. Select the interval. The interval is how often you want to check WattDepot for new data.
3. Select the sensor data type
4. Click "Monitor" this will start the action of monitoring a source and output will be displayed on the right hand side.

The current issues for the Monitor page have been added to the Issues page in our hosting site. But here's a short list of the current issues:

• Some sources take longer than the specified interval to display. Say for example the interval is set to 5 seconds. Some calls to WattDepot (i.e. a call to SIM_OAHU_GRID, the largest source) takes as long as 10 seconds to respond and output the new data. So the difference between 2 successive "Last Checked" times are longer than 5 seconds.
• Need a way to notify the user if he/she sets an interval that is way lower than the source's update. Say the selected source updates every hour, but the user selects 5 seconds as the update interval.

A nice feature would be to have a nice URL for the "Monitor" button. Right now, when the "Monitor" button is clicked, it adds the Wicket action to the URL.

E.x.

http://localhost:7070/wattdepotapps/?wicket:interface=:3:1:::

to just

http://localhost:7070/wattdepotapps/monitor

So far these are the known issues that I can think of, just waiting for tomorrow weekly meeting with Prof. Johnson and Robert Brewer to review it.

WattDepot-Apps: Second Milestone

After my groups weekly meeting with Prof. Johnson last week Tuesday, it was decided that we will continue to work on the WattDepot-Apps project for the second milestone of the semester. Our most recent distribution of WattDepot-Apps implemented the basic visualizer without multi-source capabilities, and a skeleton structure for the WattDepot Browser page. Though the distribution itself is mainly for the Visualizer while still containing source code for pages in the next iteration (i.e. Browser page).

What seemed like would be natural to start working on the Browser page, Robert Brewer and Prof. Johnson decided that the next application to be developed will be the WattDepot Monitor instead. The WattDepot Monitor is an application that will display the most recent data from a chosen source, and while the application is active, will update every so often given a chosen interval.

During the meeting we established the week's To Do list which included:

1. Re-structuring of Visualizer layout
2. WattDepot Monitor
3. Multi-source functionality to Visualizer
4. Subsource capability for Visualizer
5. WattDepot Browser

(I've also ranked them in to the most to least important)

During the meeting we also delegated on who's to do what. Since there were 2 major things to be done between the 3 of us, Kendyll would handle the re-structuring and multi-source capabilities, I would handle the Monitor page, and Bao would be the go-between.

The main focus of last week Tuesday's meeting was the Monitor, so we pretty much did a run down of how the Monitor is to be set-up. This included the layout and functionality. What I accomplished for this past week was revert the trunk to before the v1.0 Distribution. I added back in the main page so that the user can select between the Visualizer and Monitor. I've also added the Wicket UI components to the page and created an Ajax behavior that changes a label on the page to show the selected Source's update interval. Even though Bao was supposed to be the go-between and facilitate my and Kendyll's work, he showed some interest in working with the Monitor so we are going to work together in creating the page.

Earlier in the week Kendyll had moved around some components in the Visualizer page and it caused the time drop-downs to no longer work. But just recently he had corrected the error by reverting to an older version and re-arranging the layout. As of now, the trunk currently has a re-arranged Visualizer and skeleton working of the Monitor.

WattDepot-Apps: Visualizer 1.0

The time has come for the first milestone in my Software Engineering class. My group and I are to unveil the first iteration of WattDepot-Apps. What we had boldly set out to do were the three applications I described in my previous posts (Visualizer, Browser, Monitor). Most of the past month was devoted to the development of the Visualizer, a web application with a User Interface that queries the WattDepot server and uses Google Visualization's Annotated Timeline/Chart to display the data.

WattDepot-App's updated Google Code Hosting site here

*Complete with featured wiki pages for end-users as well as developers curious about WattDepot and our project.

Both the stand-alone binary .jar for the Visualizer and the complete distribution .zip file can be downloaded at our Google Code site here.

The jar file is solely for the purpose of running the Visualizer whereas the distribution contains not only the classes for the Visualizer, but also the classes that are soon to be implemented, such as the Browser. There are still quite a few issues that need to be worked out with the Visualizer page, all the issues concerning the whole project can be found at the Issues Page here.

The gist of the Visualizers Issues includes:

• Multi-select capabilities
• Subsources Option

What I want to point out is that for this first milestone, what we wanted to present was "the" most completed aspect of our work. Therefore we actually downgraded the source multi-select to a drop-down menu since the multi-select checkboxes would only take the first source selected and ignore any of the other sources. We will definitely try and get the two mentioned issues completed and start working on the browser page. Those wanting to check out the trunk via SVN may do so, but v1.0 distribution and what's currently in the trunk is different. What's currently in the trunk is current work into v2.0 which is the multi-select elements.

Looking back this past month, there were a lot of constructs I had to revisit. Most of which being Wicket and Java, but I also dealt with some JavaScript, which I never touched, I even dived into some User Interface design aspects such as menu placement, browser compliance, and resolution issues.

The hardest part of this first milestone was just getting everything to work, reading up on how to implement specific elements such as Wicket's CheckboxMultipleSelect, implement JavaScript actions to invalidate options, and working with Google Visualizations. For the most part we always knew where we were headed, we just needed to get there.

An aspect I found semi-ironic was the laying out of tasks. Before and after every meeting with Kendyll and Bao, I always made sure we knew what tasks needed to be done, however, when it came time to actually do work, I still needed to see the Issues page and constantly remind myself and the others what we needed to get done.

What I've contributed to this first milestone included setting up the initial groundwork for all development, which included creating the Google Site, setting up a Continuous Integration job through Hudson, and creating the base site for WattDepot-Apps that runs through Ant. I've also added Wicket components to the Visualizer page, as well as other HTML components that is used to query WattDepot.

The next milestone we will probably resume work on this project and continue our same work habits. I believe if we maintain our work ethics and continue to output the same level, if not higher, we can truly develop the three stand-alone applications we set out to do.

WattDepot Status: Progress Retarded

It has been three weeks since this semester began and already is the first milestone for Software Engineering coming up next week.

What I thought to have started out strong with the initial setup and progress with WattDepot-Apps, has suddenly taken a huge bump in our road to completion. Although I can't blame anyone or anything in particular, a snag was bound to happen within the three weeks of working on this project.

To re-iterate our final project deliverable, WattDepot-Apps, was set to complete three stand-alone applications for the WattDepot service. This included:

• WattDepot Visualizer
• WattDepot Browser
• WattDepot Monitor

Our main focus was to work on each project before moving to the next. We compromised that it was better to have one completed, if not, near completed project, than having 3 partially functioning one.

By the end of this week, it will mark 1/4 of the semester gone by, which means the semester is in full swing, with midterms coming up, I believe it will be even more difficult to keep up the progress we once had.

For this past week, the momentum we had died down and we stood a more steady approach to things. Our weekly meeting with Robert Brewer, the WattDepot Wiz, was canceled due to inactivity. What we set out to finish by today was a working version of Visualizer with advanced User Interface components, meaning multi-select menus for sources and data types, and the initial setup of the Browser page.

What we did accomplish though is getting the Multi-select menu to render all the sources of WattDepot, setting up JavaScript to accommodate erroneous option selection, and re-designing page layout to better suite the new User Interface components.

The problems we encountered were between the different constructs we were using to get the site functioning. The base of our site is Wicket based, which is supposed to interact well with Java, however, we also need to use JavaScript in order to get Google Visualizations up and running. Between these three systems, it was hard to jumble between each of them. The way that Wicket is designed seemed to be a hassle to code JavaScript for what we needed. The most irritating I found with Wicket was that setting Default options for menus is a complete headache. When I converted the source drop-down menu to a "CheckBoxMultipleChoice" I ended up hard-coding a default value in the JavaScript of the page because I could not figure out a simple way to set defaults using Wicket.

For this week, we will have to haul double-time. Getting the multi-source selection is our top priority, once we get that done the Visualizer is pretty much complete. For the Browser page the initial workings is in progress. Unfortunately only half of the functionality can be implemented since half is Source summaries, and the other half is User data. The User data has yet to be completed, and even then, there seems to be permission issues with viewing non-public user information.