Thomas Laubach

FabAcademy 2016, Kamp-Lintfort, Germany


ABOUT THOMAS OUR FABLAB HOME


Week 15: Networking and Communications


Homework for this week:

  1. Design and build a wired &/or wireless network connecting at least two processors


In preparation for my final project, I did following things that complete this assignment on networking (please also refer to this place).

  1. Link two Arduino compatibles via SPI. One serves as ISP programmer for the second.
  2. Attach an SD card reader via SPI to an Arduino, read a file from it and burn it on the second Arduino.
  3. Use two SPI devices on one Arduino simultaneously.
  4. I extended/modified an Arduino-as-ISP sketch such that it fulfils above mentioned functions.


Programming an Arduino with another Arduino


As my final project, I would like to build a video game system (VGS) with two ATmega1284p MCUs as its cores. I want it to read hex code from an SD card and burn the hex code on-the-fly into another Arduino's flash ROM (or Arduino compatibles). I cannot use plain Arduino Uno's for these tasks, as they do not offer the necessary amount of memory. I cannot use the Arduino Mega 2560, because I cannot solder its processor. Moreover, we need to build our own board.

In the meantime, I have verified that the LED matrix panel works with the ATmega1284p. I am now going to check whether I can upload a hex file to a second Arduino-compatible MCU using the Atmega1284p. I want this function because, from a user's perspective, this looks as if he/she can select a game and then immediately play it on my video game system without the need to use a computer.

Burning a hexadecimal file on an ATmega1284p using another ATmega

At the end of April, I did an extensive search on the WWW about about similar projects. I found an interesting Blog contribution on the Adafruit learning pages here. An Adafruit forum user has developed a program by the name Optiloader that lets an Arduino, instead of a computer, program another chip, very fast. Optiloader is very specialized, as it lets you program only one chip (not a problem for my project) using one particular HEX file (a problem for my project). The guys at Adafruit adapted Optiloader to be more flexible - so it can program any AVR with any HEX file. Another contributor, Mike Tsao, improved and extended Adafruit's further development and published the result by the name adaStandalone. One highlight among his improvements is that the memory footprint of the hex code you feed the AVR with is now roughly half as large as that achieved by the guys at Adafruit. Tsao achieved this by changing the data format. For information on what he improved, see the section "Changing the Code" on the Adafruit training website (above link) and the comments in adaStandalone.ino.

At this time I realized that I had made a notional mistake. There is a problem that is not immediately obvious: the data in the struct (see below) shall go to the Flash memory; it is not being produced while the program is running. This means that, in particular, the hex code from the SD card cannot be read in its entirety into Flash memory, where it would rest until it is written out to another Arduino. The Flash memory cannot be written during program execution; it is the same memory the program is burnt into beforehand. A solution for this is to stream the hex data from the SD card and simultaneously write it out to the second device. This is what I have to do now. An advantage would be that I could use an ordinary Satshakit for the device that is writing to the second because I would not need the larger memory of the Mighty (ATmega1284p).

But there is an alternative. With a modified Optiboot boot loader by Marek Wodzinski, it has fairly recently become possible to write to Flash memory during runtime. Wodzinski has added three new commands to the Optiboot boot loader that allow you to write pages or words or erase the memory. If my above streaming approach fails, I could also try this. The modified boot loader and the technique are detailed on this GIThub page and in this Arduino forum entry.

Modifications to the sketch AdaStandalone: For SPI, Tsao, the author of AdaStandalone, used the old syntax for commands. I changed that to the current syntax, such as "SPISettings settingsA(160000007128, MSBFIRST, SPI_MODE0)".

His program and OptiLoader (see optiLoader.h) use the following data structure to hold an image of a Intel hex-coded program in Flash memory:

  typedef struct image 
  {
    char image_name[30];  // e.g. "optiboot_diecimila.hex"
    char image_chipname[12];  // e.g., "atmega168"
    uint16_t image_chipsig;  // Low two bytes of signature
    byte image_progfuses[5];  // fuses to set during programming
    byte image_normfuses[5];  // fuses to set after programming
    byte fusemask[4];
    uint16_t chipsize;
    byte image_pagesize;  // page size for flash programming
    byte hexcode[3072];  // intel hex format image
} image_t;


I need to either fill the array hex code on the fly while I am reading from the SD card (which means I am writing to flash memory), or I bypass Flash memory and write directly to the second Arduino.

I found out that the ATmega1284p has the same page size as the ATmega328p which is used in the Arduino Uno, 256 words, which is 128 byte (see here in section "Grundlagen" and in the ATmega328p datasheet, page 297, section 25.7.2).

When I was investigating the code in AdaStandalone, I realized that it would be very hard for me to make the necessary modifications to the code for on-the-fly streaming from SD card. Also, I regarded it too risky. I decided to use above mentioned OptiBoot boot loader that lets me write into Flash memory during program execution (see comment in brackets). I installed the OptiBoot boot loader as described in the OptiBoot ZIP file I had downloaded from the Github project side. The new board definitions did not appear in my Arduino IDE 1.6.9. It worked after moving the content of the directory optiboot in a newly created folder avr. This corresponds to the way I had installed the Mighty 1284p board definitions before. The path to the folder avr is Arduino.app/Contents/Java/hardware/optiboot/avr/em>. Below this, you should find the folders boot loaders with sub folder optiboot, AtmelStudio, examples and two files with board definitions. One file is for new Arduino IDE versions (1.6+), the other one for older versions. I renamed the file with the newer boards definitions with boards.txt, and the old one boards_pre1.6.txt.

What was missing, however, was a new boot loader for the ATmega1284p. I needed to install the XCode command line tools to be able to compile a version of the OptiBoot boot loader for the ATmega1284p MCU, as this was not in the collection of boot loaders that came with Marek Wodzinski's OptiBoot. How this can be done is described in the README.TXT file that comes with his OptiBoot. I followed his explanations in section "Building Optiboot in the Arduino IDE Install". Additionally, after compiling the new boot loader, I moved it to the location that the Arduino IDE reported it would search for it. Within the Arduino IDE, I can now compile sketches that make use of Wodzinski's Optiboot boot loader, and that write to Flash memory while the sketch is being executed.

I am suffering from a new phenomenon: sometimes I can access the contents of Arduino.app from the shell, then again I cannot. I want to look at this later on.

Today I have linked the Arduino Mega 2560 to the SD card reader, read from it and tried to write the file on the Flash memory. In general, this was successful. The SD card reader I am using is an SPI device. I followed this tutorial on the Arduino website www.arduino.cc for a description on how to link the devices. As the Arduino Mega 2560 has MOSI, MISO, CLK on different pins, I used them instead: MOSI on pin 51, MISO on pin 50, and SCK on pin 52. I put SS (Slave Select) for the SD card reader on pin 53.

I wanted to do the same for the new Satshakit 128 by Daniele Ingrassia. It took hours to get the electrical connections right because, as soon as I put the SD card reader into the circuit, Arduino IDE did not recognize the Satshakit 128, or it was recognized but with a different device number, and no sketch could be burnt on it. I finally thought I had solved the problem by inserting a 10 kOhm resistor right behind the MISO pin of the SD card reader. I read about this advice in a tutorial on SPI. This still does not work.

I have decided to not use the feature from OptiBoot to load the game's hex code into Flash memory of the programming device - now an Arduino Mega 2560 - first. Instead, I have modified the program AdaStandalone such that I burn the code page per page on the target Satshakit 128 while reading from the SD card. Although I have adjusted the program in a way that it performs this task, it does not yet work. It reads from SD card but does not yet burn the fuses correctly. The Satshakit's on-board LED is attached to pin 11. The Blink example from the Arduino IDE needs to be changed accordingly. I used this sketch in order to check by wiring. I am using an Arduino Uno Rev. 3 as programmer, an SD card reader as SPI device and a Satshakit 128 as another SPI device. I wrote a sketch that allows me to read from the SD card (see below section). Using the same wiring, I could also burn a sketch on the Satshakit 128 using the Arduino Uno as ISP programmer (using the ArduinoISP sketch). Sometimes the sketch burnt to the Satshakit 128, sometimes the Satshakit 128 was not recognized, and the Arduino IDE reported device number 0x0. At one point, neither a sketch could be burnt, nor could the directory of the SD card be printed. After many tries I noticed that the Arduino IDE informed me that the Arduino Uno is low on memory and might cause unstable behavior. I switched to the Arduino Mega 2560, and I could at least read the directory contents from the SD card. But I cannot read from SD card and burn on the Satshakit using my wiring.

Working SPI with Arduino Mega 2560, an SD card reader and a Satshakit 128: On the Satshakit 128, Pin PB4 (= "Arduino" digital pin 4) is the Slave Select pin (SS), according to the data sheet for the ATmega1284p SMD package. I, too, connected pin 10 on the Arduino Mega 250 to this pin. Now two pins are linked to digital pin 10 of the Arduino Mega 2560: pin PB4 and pin RST of the Satshakit. Now I can both access the SD card reader and burn sketches on the Satshakit using the same circuit (see below image). I have also added a 100 uF capacitor to disable the autoreset function on the Arduino Mega 2560 (I read this hint in an Instructables tutorial).

I inserted a series 10 kOhm resistor to the MISO pin from the SD card reader. I did that because this topic from electronics.stackexchange.com raised following point:
"You have to prevent any other device from driving the SPI lines during programming. To avoid driver contention, a series resistor should be placed on each of the three dedicated lines if there is a possibility that external circuitry could be driving these lines." The result was that the SD card reader was no longer recognized. The program could no longer initialize it. But it had the pleasant side effect that the fuses of the Satshakit 128 were burnt successfully:

    
Initializing SD card...initialization failed!
initialization done.
Searching for image...
  Found "Blink.ino.hex" for atmega1284p
Starting Program Mode
In start_pmode: spi_transaction
did spi_transaction
 [OK]

Setting fuses
  Set Lock Fuse to: 3F -> ACE000  Set Low Fuse to: FF -> 3FACA000  Set High Fuse to: DE -> FFACA800  Set Ext Fuse to: 5 -> DEACA400
Verifying fuses...
	Lock Fuse: 3F is 3F	Low Fuse: 0xFF is 0xFF	High Fuse: 0xDE is 0xDE	Ext Fuse: 0x5 is 0x5


This lead me to the conclusion that I had got something right. I would need to tweak the SPI settings in my code. After the first use of the SD card reader I inserted an "SPI.endTransaction(), and before the first access to the Satshakit 128 via SPI I put a "SPI.beginTransaction(settingsA)". SettingsA consists of the following parameters: clock for SPI: 16000000/128, transmit MSB first, use SPI_MODE 0. After some time, however, it turned out that I can only reliably burn the fuses if I do not initialize the SD card reader. Again, more investigation is needed.


Fig. 1. Working SPI connections: Arduino Mega 2560 as ISP programmer and SD card reader


The games on the SD card are saved in Intel hexadecimal format. I want to convert them into binary files. This would allow me to write the byte stream page per page onto the target Satshakit 128. On a Windows 10 PC, I used the program srecord v1.64 from here to convert them from Intel hex into binary. This can quite simply be done from the Windows prompt via "srec_cat.exe BLINK.HEX -intel -o BLINK -binary".

I am again suffering from a strange error: sometimes my sketch is burnt on the Arduino Mega 2560 without problems, sometimes the Arduino IDE reports a timeout ("avrdude: stk500v2_ReceiveMessage(): timeout") like here:

    avrdude: AVR device initialized and ready to accept instructions

Reading | ################################################## | 100% 0.01s

avrdude: Device signature = 0x1e9801
avrdude: reading input file "/var/folders/37/bk665qlx7rq8s2r4bq8sq7sr0000gn/T/buildce151a04b7de51f214d21617af47daf0.tmp/burnFileFromSDCard.ino.hex"
avrdude: writing flash (21064 bytes):

Writing | avrdude: stk500v2_ReceiveMessage(): timeout
    

This occurs even with a brand new Arduino Mega 2560 with no wires attached, and right after I rebooted my MacBook Pro. I also tried different USB cables without positive result. I have found a very strange workaround: it works when I copy and paste my main Arduino sketch file into a fresh file and copy it into a new folder which of course bears the same name as the Arduino sketch. Any clue why this works? Does my program garble the contents of my folder? Edit: apparently, this trick works occasionally. Newer edit: I cannot burn to my Arduino Mega 2560 at all any longer. The same occurs with my Arduino Uno and a SainSmart Mega 2560, on my Windows 10 PC and on my MacBook Pro.

A user on this forum reports problems that hinder the collaboration between the SD card library by Arduino and Sparkfun and the SPI library. He/she recommends to use the ancestor of the SD library, SdFat library by someone named Greiman. You can find it here. I will try this after FabAcademy, as I am running out of time.

For my video game system, sadly, this means that it will not have the attractive features I wanted for it. The player will not be able to load a game from SD card. Instead, for the time being, the system will remain a toy for advanced users.


Reading files from an SD card

I am using the (Arduino) SD card library that comes with the Arduino IDE installation to read (Intel hexadecimal) files from an attached SD card reader. The SD card library supports only 8.3 format files. The SD card library makes it simple to read files from and write to an SD card. I attached my SD card reader to the Arduino Mega 2560 using the following connections:

    GND (Arduino Mega 2560) --- GND (SD card reader)
    5V  (Arduino Mega 2560) --- VCC (SD card reader)
    50  (Arduino Mega 2560) --- MISO (SD card reader)
    51  (Arduino Mega 2560) --- MOSI (SD card reader)
    52 (Arduino Mega 2560) --- SCK (SD card reader)
    53 (Arduino Mega 2560) --- CS (SD card reader)
    


Fig. 2. The SD card reader I am using


Pin CS on the SD card reader serves as the Slave Select pin in the context of SPI.I prepared an SD card following the advice from here (file system FAT32, names not longer than 8 characters) with the hex code of games I have written over the time. I used this example program to print out its contents:

Initializing SD card...initialization done.
BLANK.hex
1453
BLINK.hex
3417
GIGAPEDE.hex
73617
INVADERS.hex
91302
PACER.hex
125015

Where do the files come from? The Arduino IDE temporarily stores the hexadecimal representation for a sketch it is compiling in a temporary directory with a seemingly random name. The version of the IDE I am using (1.6.9) does display the path to the hex file during compilation. I copied the file from the respective location.

To circumvent problems with my SPI devices, I will switch to the library SdFat (see above comment).


Serial communication between two Arduino compatible boards


To close my assignment on Networking, I have decided to do something straightforward: I take my sketch from week 16, Applications and Implications, and let it talk to the computer as before, but with the detour over the Satshakit I built some time before. My VGS board for my final project sends the joystick movements over a hardware serial port to my Satshakit that receives over a software serial port. The Satshakit sends the exact data it receives over its hardware serial port to the computer.

I used MISO/MOSI/SCK pins to burn the boot loader on the boards using an Arduino Uno as ISP programmer. Then I uploaded the first sketch to my VGS board that would act as transmitting device and the second sketch to my Satshakit that would be the receiver of the joystick input. After the sketch was on the Satshakit, I attached to it an FTDI cable that was linked to my computer. Without FTDI cable (and FTDI chip), no serial connection between the computer and the Satshakit would have been possible. For testing, the Arduino IDE must use the serial port that is linked to the Satshakit in order to display the correct data in the serial monitor. The Satshakit uses its hardware serial line to mirror everything that it is receiving across the software serial line from the VGS board straight to the computer. For serial communication, pin PD0 (RX0) is connected to pin 6 on the Satshakit, and pin PD1 (TX0) on the VGS board is connected to pin 5 on the Satshakit. These digital pins can be selected freely for software serial communication. The sketch on the Satshakit writes exactly the bytes on the serial line to the computer it has received from the VGS board: I use Serial.write() instead of Serial.print().

Apart from different digital pins used for polling the joystick, nothing changed in the sketch on the VGS board in comparison to the code presented in the assignment on Applications and Implications.


Fig. 3. The two boards I made ready for communication across a serial line


As the sketches are very short, I can reproduce them here:

Sathsakit

#include 

SoftwareSerial mySerial(5, 6); // RX, TX

void setup() 
{
 Serial.begin(9600); // I use this serial line for communication with the computer
 Serial.println("I am here");
 mySerial.begin(9600); // this serial line communicates with the other Satshakit
}

void loop() 
{
  if (mySerial.available() > 0)
  {
    Serial.write(mySerial.read());
  }
}


VGS board

// Author: Thomas Laubach

// Joystick
int joyUP = 11;
int joyDOWN = 12;
int joyLEFT = 13;
int joyRIGHT = 14;
int joyFIRE = 15;

// Digital signals to send for each joystick/button movement
int serialUP = 0;
int serialDOWN = 0;
int serialLEFT = 0;
int serialRIGHT = 0;
int serialUPRIGHT = 0;
int serialUPLEFT = 0;
int serialDOWNRIGHT = 0;
int serialDOWNLEFT = 0;
int serialFIRE = 0;

byte t;

void setup()
{
  // Initialize serial connection
  Serial.begin(9600);
  Serial.println("Hello World!");
  
  // Initialize joysticks and  buttons
  pinMode(joyLEFT, INPUT); // DOWN
  pinMode(joyRIGHT, INPUT); // UP
  pinMode(joyUP, INPUT); // LEFT
  pinMode(joyDOWN, INPUT); // RIGHT
  pinMode(joyFIRE, INPUT); // FIRE

  // Activate internal pull-up resistors
  digitalWrite(joyLEFT, HIGH);
  digitalWrite(joyRIGHT, HIGH);
  digitalWrite(joyUP, HIGH);
  digitalWrite(joyDOWN, HIGH);
  digitalWrite(joyFIRE, HIGH);
}


void loop()
{ 
  if (t == 255) { t = 0; } else { t++; }

  if (t % 10 == 0)
  {
    if (digitalRead(joyFIRE) == LOW)
    {
      serialFIRE = 1; 
    }
    else { serialFIRE = 0; }
    
    if  (digitalRead(joyUP) == LOW) // NORTH
    {
      serialUP = 1;
      
      if  (digitalRead(joyRIGHT) == LOW)
      {
        serialUPRIGHT = 1;
      }
      else { serialUPRIGHT = 0; }
      
      if (digitalRead(joyLEFT) == LOW)
      {
        serialUPLEFT = 1;
      }
      else { serialUPLEFT = 0; }
    }
    else { serialUP = 0; }
    
    
    if (digitalRead(joyDOWN) == LOW) // SOUTH
    {
      serialDOWN = 1;
      
      if  (digitalRead(joyRIGHT) == LOW)
      {
        serialDOWNRIGHT = 1;
      }
      else { serialDOWNRIGHT = 0; }
      
      if (digitalRead(joyLEFT) == LOW)
      {
        serialDOWNLEFT = 1;
      }
      else { serialDOWNLEFT = 0; }
    }
    else { serialDOWN = 0; }
    
    if (digitalRead(joyLEFT) == LOW) // WEST
    {
      serialLEFT = 1;
    }
    else { serialLEFT = 0; }
    
    if (digitalRead(joyRIGHT) == LOW) // EAST
    {
      serialRIGHT = 1;
    }
    else { serialRIGHT = 0; }

    // Send the data through the serial line
    if (t % 10 == 0) // Do not pollute the serial line
    {
      Serial.print(serialLEFT); Serial.print(",");
      Serial.print(serialRIGHT); Serial.print(",");
      Serial.print(serialUP); Serial.print(",");
      Serial.print(serialDOWN); Serial.print(",");
      Serial.print(serialUPLEFT); Serial.print(","),
      Serial.print(serialDOWNLEFT); Serial.print(",");
      Serial.print(serialUPRIGHT); Serial.print(",");
      Serial.print(serialDOWNRIGHT); Serial.print(",");
      Serial.println(serialFIRE);
    }
  }
}
    


Source files

Arduino sketch(es) that stream game code from SD card and burn it to second Arduino compatible device: here
The example program for the SD card reader: here
SDFat library: here
Arduino sketch for the Satshakit, see section on serial communication between two of by boards: here
Arduino sketch for the VGS board, see section on serial communication between two of by boards: here
Link to the files and the descriptions for my modified Satshakit 128 board: here

Lessons learned