Monday, November 1, 2010

[Data In] Connect Arduino to the Cloud

Here is a fun project to log Arduino data directly to the Google Cloud using your local network without any middle tier. That is, your data goes right from the Arduino and then out to Google to be logged as a Nimbits Data Point.

You may want to download and setup a copy of the Nimbits Data Logger Server on your own app engine account. Learn More Here

Nimbits is a free, open source data logging server. You can use the public server (with some limitations) on app.nimbits.com or setup your own running instance.

The circuit is very simple and only requires an LM35 temperature sensor and three wires.  I went as far as to waterproof my LM35 by soldering it to the end of a cat5 cable and then sealing it in shrink wrap tubing. This is then connected to the ground, 5v and analogue pin 0.  There are many examples of this circuit online like this one: http://pscmpf.blogspot.com/2008/12/arduino-lm35-sensor.html

Here is mine using the Ethernet Shield and my waterproofed LM35



With the code below, i'm sending temp reading directly into a nimbits data point. Since Nimbits integrates into the google chart api, i'm able to generate all kind of charts with this data that I can then embed into emails, web pages, docs etc. Like this:


This chart really is the current reading from my Arduino device, since the image is being loaded from the Nimbits Chart Service.

Here is all of the Arduino code you need to feed your device readings into the Cloud.  You'll just need to modify the service url with your server secret (if you are running your own server) or make your point public, and provide an email and user key.

This code posts the temp reading to google with the host header nimbits1.appspot.com. If you are using the public nimbits server then you keep that. If you set up your own copy of Nimbits on your app engine account. You just replace nimbits1 with your app id.

Because we are proving the host header, we can send this request to google.com's IP and our request will be relayed to the correct service. From there, we can configure nimbits to relay out data to any other url, facebook or twitter.


#include <SPI.h>
#include  <Ethernet.h>
 
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
byte ip[] = { 192, 168, 1, 17 }; // a valid IP on your LAN
byte server[] =  {72, 14, 204, 104}; // Google.com or Appspot.com's IP
Client client(server, 80);
int pin = 0; // analog pin
int tempc = 0,tempf=0; // temperature variables
int samples[8]; // variables to make a better precision
int i;


void setup()
{
  Ethernet.begin(mac, ip);
  Serial.begin(9600);
}
void loop()
{
  for(i = 0;i<=7;i++)
  {  
    samples[i] = ( 5.0 * analogRead(pin) * 100.0) / 1024.0;
    tempc = tempc + samples[i];
    delay(1000);
  }
  tempc = tempc/8.0;  
  tempf = (tempc * 9)/ 5 + 32; 
  tempc = 0;
  String s = String(tempf,DEC);
  if (client.connect()) {
     client.println("GET /service/currentvalue?value=" + s + "&point=test&email=bsautner@gmail.com&format=double&secret=yournimbitssecet HTTP/1.1");
     client.println("Host:nimbits1.appspot.com");  //here is your app engine url - app id with appspot.com
     client.println("Accept-Language:en-us,en;q=0.5");
     client.println("Accept-Encoding:gzip,deflate");
     client.println("Connection:close");
     client.println("Cache-Control:max-age=0");
     client.println();
     client.stop();
  } 
 
} 
 
Enjoy! 

20 comments:

  1. looks very promissing ... not clear though on first sight how to get yournimbitssecet for posting data to nimbits1.appspot.com.

    thanks a lot,
    au

    ReplyDelete
  2. apparently was trying to post data to nimbits1 host instead of deploying my own nimbits instane in appspot. finally got it working. great piece of sw!
    (now trying to get charts integration working)

    cheers,
    au

    ReplyDelete
  3. Having trouble posting a value to a data point on my Appspot - I keep getting bad request responses from the server. I know it has something to do with my code, which is different that what you have above since I'm using a WiShield.

    Also, the Nimbits manual isn't quite clear how to do a POST, which is what I'd think I'd be doing, versus your GET above. Not sure how you're posting a value with a GET, or with a "currentvalue" - the documentation doesn't reflect either of those methods. Is it out of date? If so, is there a current reference?


    -----------------

    WiShield code for POST:
    POSTrequest(ipAddr, port, hostName, URL, bodyFunc)

    The parameters per the Nimbits manual:
    key, pointname, owner, Timestamp, and value

    In the URL I'm specifying key, pointname, and owner since those values aren't changing (for now). In bodyFunc I'm calculating the Timestamp and value.

    Is there a way to POST a value to a pointname, but have the Nimbits server use its current Timestamp? Would save me having to calculate that value, which would be more convenient even if it meant a loss of accuracy.

    ReplyDelete
  4. Hi Joel,

    I think we modified the currentvalue service to also record data with the GET because of difficulties getting POST to work in arduino ethernet.h. POST works for this service, but we hacked GET to look for input values to make life easier.

    I'm going to take a harder look at that. But at first glance here are my thoughts

    appspot.com/service/currentvalue will accept a post for new values. If timestamp is not provided, it will use the current UTC time.

    the POST needs some level of authentication along with it. I wouldn't want to try to get the auth cookie from google in c, make sure you're providing a secret parameter and email so the system can identify you.

    something like this:

    /currentvalue?point=pointname&value=1.23&email=test@example.com&secret=user-or-server-secret.

    you can also try url encoding the parameters by hand, i'm not sure if that's an issue here. (the @ in email for example)

    Since you're running your own server, try using the server secret that is in your datastore as a serversetting entry.

    It's vital that you set
    client.println("Host:nimbits1.appspot.com");
    with your app id in the arduino code so google knows where to relay the post.

    Hope that helps.

    ReplyDelete
  5. Hi, I found out that somehow the concatenation of strings didn't work. So I am using the PString library now with

    PString str(buffer, sizeof(buffer));
    str.print("GET /service/currentvalue?point=test&value=");
    str += data;
    str += "&email=***&secret=*** HTTP/1.1";
    client.println(str);

    don't know why, but took me a while to find out, so this could be helpful to others..

    ReplyDelete
  6. I like the idea idea to log data. But it's seems useless if i can't put in relation with Time. In your sketch if i am correct you send every second the average of all the analog input data create during that second.

    I don't make the relation with the #7 in the for()loop.

    anyone help Thanks.

    Alexandre

    ReplyDelete
  7. Hi Alexandre,

    I was looping through 8 temp readings from the LM35 and then taking the average, just to see if I could smooth out the reading for accuracy purposes. I suppose you really don't have to do that. You can take just one reading and send it right to a data point.

    If you don't supply Nimbits with a timestamp, it takes the current time the reading was submitted, but you can also submit a specific timestamp on your end and nimbits will record the value at that time.

    I hope that answered your question, please let me know if I can be any more help. Check out www.nimbits.com and the discussion forum too on http://groups.google.com/group/nimbits


    - Ben

    ReplyDelete
  8. do you think this could work with 10 temperature inputs? If so, how would I change the code?

    ReplyDelete
  9. Hi Christian, sure, this would work with any number of inputes, you would just log into app.nimbits.com and set up 10 data points and then edit the code to take the separate readings. You could use the batch service to record all 10 readings at once. Check out the API Overview doc on https://code.google.com/p/nimbits-server/ that would show how to do that sort of http post.

    Please post the result or your progress to our discussion forum, I'd like to see this work

    http://groups.google.com/group/nimbits

    ReplyDelete
  10. I will thanks, I have 4 acs in my house. I want to log input output temps so I can track filter or freon issues. Also put one in the pool heater output. I think it could also be interesting to hook a current coil output to one to monitor whole house usage. Just need to find the time.

    ReplyDelete
  11. .net not working ---404 error

    ReplyDelete
  12. Not working for me neither. I have a timeout for Ethernet. I can connect to any website except App Engine. How is that possible?

    ReplyDelete
  13. Your example sketch doesn't appear to work, at least not in Arduino 1.0. When you verify the sketch you receive the error message:

    'Client' does not name a type

    and the fifth line:

    Client client(server, 80);

    is highlighted.

    I was also wondering if the first line is suppose to be:

    #include >

    Any ideas? I'm looking forward to getting the example working. Thanks!

    ReplyDelete
  14. Hi Cris, i didn't realize the includes weren't showing up, thanks I fixes the post. A lot of users have made use of this code sample in Arduino 1.0. Let us know how you make out. Check out out user forum, lots of users on their have used this code.
    https://groups.google.com/group/nimbits

    ReplyDelete
  15. Thanks for updating that but unfortunately I'm still getting an error message when I try running the script. Have you tried cutting and pasting the script and running it? I get the following error message:

    The Client class has been renamed EthernetClient.

    and the line of code "Client client(server, 80);" is highlighted.

    ReplyDelete
  16. Check out this thread in the discussion forum, another user had this issue but got online just by switching out EthernetClient for Client - some change in the ethernet library:

    https://groups.google.com/group/nimbits/browse_thread/thread/e08e05975b21d9bb/13690a7ae3e6e042?lnk=gst&q=EthernetClient#13690a7ae3e6e042

    ReplyDelete
  17. Yep, I just finished using his code and it works! I have an issue where the temp updates when I cycle power on the Arduino but then it doesn't update after the initial one. I'm not sure why yet but I'm glad to know the code works. Hopefully the next user who tries this with the Arduino 1.0 IDE will read these comments so they can find the modified sketch that Jim wrote. I assume your code above runs on the pre 1.0 IDE? I should give it a try. Thanks..

    ReplyDelete
    Replies
    1. I thought I would post my progress so far. Using the new modified sketch I was never able to get more than one temperature update without cycling power so I decided to try the original code. Using the Arduino 023 IDE the code from this blog verifies and compiles just fine. I was just never able to get it to show up in Nimbits. I give up for now.

      Delete
    2. I went ahead and used the code mentioned above in the link to the discussion forum and uploaded it using the Arduino 1.0 IDE. The sketch compiles and the temperature will update to Nimbits but it will only do it once. The only way to get the temperature to update again is to reset the Arduino board. How are others getting this code to work? Does anyone have a working example they would like to share?

      Delete
  18. Hi Benjamin

    I am quite new at the Arduino, let alone the code.
    I am trying your sample code (listed above) but also ran into the issue "Client client(server, 80);" that Chris mentioned above. I modified the code best I understand and what I could find via the threads.

    I am now at a point where the code compiles( not sure if the code is correct), but have difficulty figuring out how to connect and how to set up the code so that I can see my data on the server.
    The problem may be that the code compiles but I don't connect or that I connect but am to stupid to set up the my details correctly on the server as to read the arduino ...:)

    Could you possibly see if the code makes sense (as far as authentication and connectivity to the server goes)?

    Kind regards

    Here is the code:


    #include
    #include

    byte mac[] = {0x90, 0xA2, 0xDA, 0x0D, 0x53, 0xD8 };
    byte ip[] = { 192, 168, 0, 14 }; // a valid IP on your LAN Office
    IPAddress server[] = {72, 14, 204, 104}; // Google.com or Appspot.com's IP
    EthernetClient client;// This may be incorrect as I modified your code here
    int pin = 0; // analog pin
    int tempc = 0,tempf=0; // temperature variables
    int samples[8]; // variables to make a better precision
    int i;


    void setup()
    {
    Ethernet.begin(mac, ip);
    Serial.begin(9600);
    }
    void loop()
    {
    for(i = 0;i<=7;i++)
    {
    samples[i] = ( 5.0 * analogRead(pin) * 100.0) / 1024.0;
    tempc = tempc + samples[i];
    delay(1000);
    }
    tempc = tempc/8.0;
    tempf = (tempc * 9)/ 5 + 32;
    tempc = 0;
    String s = String(tempf,DEC);
    if (EthernetClient()) {
    client.println("GET /service/currentvalue?value=" + s + "&point=Temp1&email=christo@eirenehealthshop.co.za&format=double&secret=temp1key HTTP/1.1"); // Not sure if I have these correct, what should my secret be or where do I set or select or copy it, is it maybe the UUID
    client.println("Host:nimbits1.appspot.com"); //here is your app engine url - app id with appspot.com
    client.println("Accept-Language:en-us,en;q=0.5");
    client.println("Accept-Encoding:gzip,deflate");
    client.println("Connection:close");
    client.println("Cache-Control:max-age=0");
    client.println();
    client.stop();
    }

    }

    ReplyDelete