Today was the last day of research! Sorry for the rapid drop in posting; things got really busy in the last couple of weeks.
To sum the experience of summer research, I would say it’s a lot of fun and work, and definitely worth it! Especially if you got to spend it with Chairman Nao 🙂 A lot of cool things were done, and new skills gained, and lots discovered. A short list of life lessons I learned in these 10 weeks:
- Debugging under time pressure should be avoided at all costs. I think we should host a tournament for this and see who breaks sanity the fastest.
- Humans are unpredictable, and imitating them is quite difficult.
- Dual screen displays are beautiful things, and my laptop (on which I write these words) pales in comparison.
- The joy of bagels in the morning, and my deep sadness when I realize there’s no more cream cheese left.
- Hardware fails more frequently than you want it to.
It’s been a great summer, and thanks to everyone who was part of it! To end this post, here’s a video of Chairman Nao being Chairman Nao:
In the past few days we’ve been hard at work getting the application into a good place for the end of summer. Our goal is to hopefully send the application home on the phones of some summer researchers to get feedback, find any bugs that are lurking, and in general have a sense of how things will go during the semester.
PaWPal in the wild! (Well, almost–it’s running on a phone we borrowed from the department.)
In order to have PaWPal in tip-top shape, we’ve been writing a lot of code to do various things. Server code now lets us send and save all kinds of data, from pings to a user’s calendar events and tasks, and we’re almost done fixing the bugs that were interfering with push notifications. Initially, we couldn’t get any to send, then we had too many being sent at the wrong times, but it seems that we’ve got it (almost) fixed!
In addition, we’ve gotten more visualizations working, and have more planned to put in before we end research next week. The new visualizations will show a user different information about their emotions, flow metrics, and more. We’ll be wrapping up for the summer really soon, so it’s back to work for the PaWPal team!
Many things have changed over the past days and weeks in the Robot Brunch project. The good news is that we can now easily run decoupled experiments on the robots, such as the robot-swap experiment in the video below.
The other news is that s we considered the preliminary results of our decoupling tests on random networks, we realized that the timing of node execution is more important than we had realized. In some of the random schedules we generated one of our decoupling algorithms managed to do significantly better than both the brute force decoupling and even the original undecoupled schedule. After investigating, we discovered that in these schedules the optimal strategy was not to execute as soon as possible but rather to wait for another agent to make some amount of progress. In the example below, if the first agent executes timepoint 1 as soon as it is allowed it will fail timepoint 2 almost every time.
This was the end of decoupling, since to be fair to the algorithms we needed a robust way to execute the schedules that could be applied in simulation and practice to the decoupled and undecoupled versions.
Fortunately, we found we could easily re-purpose the one algorithm that realized it needed to wait and use it for execution. Along with another algorithm we developed, the execution strategy could be re-evaluated many times during execution to breathe life into even the most tangled schedules. We read about controllability and execution theory to improve our algorithms, and decided this is what we needed to discuss in the paper.
Today we broke and fixed the algorithms. Then we broke and fixed them again. By trying several simple variations of a clever problem, we were able to find many cases that were not handled by our old decoupling algorithm or the new execution strategy based on it. While we were able to fix the execution strategy, the decoupling version is still incomplete. Much work remains as we wrap up the second to last week of research. Writing and testing needs to be almost finished by the end of next week.
This week, we’ve been working hard on our paper and IRB drafts. We even spent all of this morning writing while a projector was being put in our lab. Our team also met yesterday and revised our ping/survey page with all new questions! We’ve also been working hard to get our server up and running.
Next on our agenda is to keep on writing and get the server to talk to our application even when they’re not on the same wifi network. Once this works, our app will be able to communicate and get information to and from the server even when we aren’t in the lab. We also need to make sure our surveys send the right information and that everything is private. There’s still a lot to do, but I’m confident we’ll get it done before the end of the summer!
Here are some pictures of our updated survey/ping page!
This week we are making the final push on several fronts…for real this time. It often feels like the last 5% of the project takes a disproportionately massive amount of time, which is definitely what I’m feeling as I write this. We actually underwent a shift in the experimental setup, which will better measure what we’re after. However, this does mean minor tweaks in the IRB application, the HIT, and the code. It’s an infinitely iterative process! We hope to get at least a draft of the application to the IRB tomorrow morning, pilot the HIT with lab members, and get all the pieces together soon. Lesson learned: human subject studies are difficult.
For an uplifting photo, we went to Costco to restock the CS kitchen (and our minifridge)(and my otterpop needs). Prof. Dodds appeared before us in the frozen food aisle and handed us a gift card, it was magical. Here’s Kyle with a sampling of what we got…
Work continues on both the front end and back end of the application. We’ve got some really cool visualization tools in use that will let us show users interesting and (hopefully) informative data about themselves! In addition, we’ve finally got the application talking to the server’s Flask code, which then does the talking to the database. The database then passes info to Flask, which gives the app JSON data that can then be utilized however we see fit.
Flask holds the conversations, the database and application send and receive the data.
Some neat clock and pie charts that we’ve been playing around with.
We’ve also made some headway on paper writing and drafting up our IRB application so that we can run a trial study in the fall. More of the same is on the horizon for next week–more than enough time for more fun bugs to appear! We’ve overcome some coloring and server connection issues already, and are ready to see what new issues pop, waiting to be solved.
After much work on a handful of algorithms this summer, we are starting to switch gears and put them all to the test. We have been worried we’re giving our algorithms ‘easy’ problems, so we developed a way to generate random schedules! We can change how many agents, how probable it will be to create inter-agent edges, and many other factors. Once it’s done we get a nice schedule like this:
Our next step is running all of our algorithms over many random schedules to determine how robust they are, and hopefully we’ll get some insight into how to improve our current methods!
A quick note before we resume our regularly scheduled programming: tour groups are great! We had elementary and high school groups come in to look at the lab, and their excitement about robotics/AI/CS in general was awesome! It’s really cool to see people of all ages come in to look at the things we’ve been doing so far.
Anyway, here’s the update! I’m troubleshooting the HIT in MTurk’s Sandbox right now, and there are a couple unexpected issues that I have to iron out. I’m also concurrently working on the physical setup for the experiment, which involves coding the PD strategies, movements, and object recognition into the NAO. The pieces are largely complete so far, and now I have to combine them into a cohesive behavior. The IRB approval form is undergoing some initial revisions and I hope to have it submitted by tomorrow. The NAO has some pretty cool capabilities, which utilize a camera with enough definition to do expression analysis. Below are the resulting screenshots… unfortunately, it was able to discern my gender with only a 7% confidence level. Not sure what that says about me.
Exciting things to come in the next week! Stay tuned!
Over the last few days we’ve been working hard to implement the first parts of our application. While it’s not quick work, we’ve made some really good progress, as you can see in these screen shots.
Part of our survey
More of our survey
The Task List
The Navigation Drawer
Today, we had a group of high school students come into our lab and look at our projects. They said that they had trouble planning their schedules too and seemed really interested in our application. Other than that, we’ve been working on the paper draft and continuing to make the app all day. We also spent some time prepping for a meeting we are having with another lab tomorrow. It’ll be really cool to hear about new and exciting projects!
For the next few days, our plan is to continue working on our paper and IRB draft, as well as set up our own server for the application.
For the past few days we have been working on running Robot T experiments and developing timepoint distributions. We can run 100 experiments per hour, but the distribution of the data changes over the course of several days. We need to find out why this is happening so we don’t bias our results.
We have been writing a simulator that takes our old edge distributions and shifts them to timepoints instead. Then hopefully we can come up with a good decoupling algorithm out of this. So far, we seem to be able to get an optimal decoupling when there is only one inter-agent edge.
More visitors have also been stopping by, including school groups brought by the MyCS team. We have been giving explanations and demonstrations of our work and our robots.
We only have four weeks left, so we are trying to finalize our algorithms and get all the empirical data we can.