Skip to content

PROGMEM: the gift that keeps on giving

If you program on an Arduino enough or start with projects that require a lot of memory you may come across something that looks like the following:

const char string_0[] PROGMEM = "String 0";
const char string_1[] PROGMEM = "String 1";

That stores those strings, arrays of char variables really, in the program space of the Arduino. That’s right, in the same space as your program sits. The Arduino has two kinds of memory space: program space that stores your program that you upload to the Arduino and the SRAM in which the program that you upload to the Arduino initializes variables and changes values. The Arduino IDE calls these the program space and the dynamic memory:

Screen Shot 2016-04-27 at 17.17.57

The dynamic memory in particular is important to your application since you don’t know how much of it will be used by a program and it can lead to weird and hard to track down crashes or behaviors. Dropping large blocks of data like arrays of strings or bitmaps or masks for images into the program space means that you have more SRAM space for your program to run in. That’s all pretty well documented but what isn’t all that well documented in how to do more with that program space than grab a variable or two out of it. We’re going to dig in a little bit deeper and see what we can find.

Program Memory vs SRAM vs EEPROM

Here’s a very quick refresher on the difference between the 3 kinds of memory we have on an Arduino:

illustration

In order to make sense of this whole PROGMEM thing, we have to discuss the Harvard Architecture. For micro-controllers it’s a great way of organizing memory and resources efficiently in small spaces that keep the instruction set small and the programming interface simpler. The only drawback is that it’ll remind you that it went to Harvard at least once a day. You’re probably used to a Von Neumann architecture because that’s likely what runs on your laptop/phone/tablet that you’re reading this on (ok, well CPUs today have both Harvard and Von Neumann elements but they’re more Von Neumann and it’s easier to call them Von Neumann). In a Von Neumann architecture your program and your memory live in the same memory space, meaning that your program can rewrite itself while it runs and use different kinds of memory space for all sorts of tricky things. That also makes the chip and its instruction set more complex, which is less than ideal in a tiny chip with a small memory footprint. In the older Harvard architecture, the program space and the memory space that the program uses both live in separate parts of memory. That’s why when you upload some code to your Arduino you see the two sets of memory that you saw above. Things that won’t change in your program are written to PROGMEM, for instance:

const int WONTCHANGE = 5;

That’s going in PROGMEM, while:

int MAYBEWILLCHANGE = 5;

That will go into SRAM because it can change, any function can come along and set MAYBEWILLCHANGE to 4 or -1000. That should be a hint of the vital difference between PROGMEM and SRAM: you can’t change anything in PROGMEM. Ever. Unless you’re wanting to get into some seriously dark arts. If you do manage to ever get this running on a friendly Harvard architecture board (i.e. an Arduino), drop a line, I’d love to hear more about it. Now, there’s one technical caveat that we have to address and that is that the Arduino (and the Atmega and ATTiny chips that run them) are actually ‘Modified Harvard Architecture’. In a true Harvard Architecture SRAM and Program memory can’t communicate with one another at all. That would mean no PROGMEM and that would mean less fun stuff that we could do with our Arduino. Most modern computers that are documented as Harvard architecture are actually using a Modified Harvard architecture, which means that they do have separate instruction memory (Program Memory) and data space (i.e. SRAM). Ok, let’s get back to the fun stuff.

Accessing Program Memory

The canonical way to get stuff out of PROGMEM is to use a method like pgm_read_word and pass it the array that you’re accessing along with a subscript:

  const char greeting[] PROGMEM  = {"Hello from Teague Labs. This is our PROGMEM tutorial."};

  void setup()
  {
    Serial.begin(115200);
    int length = strlen_P(greeting);
    for( int i = 0; i < length; i++ )
    {
      Serial.print( (char) pgm_read_byte_near(greeting + i));
    }
  }

pgm_read_word() grabs a byte from an array and drops it into SRAM where you can do things with it like print it to the Serial console. There are a wide range of methods for grabbing memory out of the PROGMEM space and dropping it into the SRAM, methods that probably look familiar to you if you’ve spent time doing much C programming: memcpy_P, strcat_P, memcmp_P, except that they all copy data from the program memory into SRAM. What this means is that you can work with memory in the ways that you might be used to from C. One of the most helpful things in working with memory is being able to work with locations in memory, pointers. Helpfully, there’s a pointer type for memory in the program space that works just the same way that a normal pointer works, you can increment it, read it from anywhere: PGM_VOID_P. This is particularly useful when you want to walk through big arrays of data and keep track of where you were, if you have, for instance, a large block of bitmap data, waveforms, or lookup tables that you’ve calculated.

Another handy use to put our PROGMEM to use to is storing structs. As your Arduino communicates with more things and needs to play more roles can come in quick handy, for example:


struct Arduino {
  byte type;
  char port[64];
};

PROGMEM const Arduino arduinos[] = {
  { 3, "/dev/port1\0" },
  { 15, "/dev/port2AndSomeReallyLongExtension\0" }
};

void setup() {

  Serial.begin(115200);  
  // get a pointer
  PGM_VOID_P ptr = (void*) pgm_get_far_address(arduinos[0]);
  // the beginning of our struct is the first field
  Serial.print("Arduino 1 number is ");
  Serial.println(pgm_read_byte(ptr));
  ptr = ptr + 1;
  // just read the whole string length in, nbd
  int arduino1PortLen = strlen_P( (const char*) ptr);
  Serial.println(arduino1PortLen);
  // now copy it into SRAM so we can use it (or don't)
  char port1[arduino1PortLen];
  strcpy_P(&port1[0], (const char*)ptr);

  for( int i = 0; i < arduino1PortLen; i++ ) {
    Serial.print(port1[i]);
  }
  Serial.println();
  // now go to the next Arduino struct
  ptr = (void*) pgm_get_far_address(arduinos[0]) + sizeof(Arduino); // now go to the next 
  // Repeat
  Serial.print("Arduino 2 number is ");
  Serial.println(pgm_read_byte(ptr));

  ptr = ptr + 1;

  int arduino2PortLen = strlen_P( (const char*) ptr);
  Serial.println(arduino2PortLen);

  char port2[arduino2PortLen];
  strcpy_P(&port2[0], (const char*)ptr);

  for( int i = 0; i < arduino2PortLen; i++ ) {
    Serial.print(port2[i]);
  }
  Serial.println();
}

Where is PROGMEM?

Another funny thing about the Arduino PROGMEM is that it’s at the beginning of your Arduino program. So when you put the number 1 into PROGMEM at the start of Blink.ino, the program memory of the Arduino actually consists of a ‘1’ followed by the bytecode for Blink.ino. You can actually dump your entire program to the Serial:

#include <avr/pgmspace.h>
PROGMEM static const uint8_t charSet  = { 0xF0 };

void setup() 
{

  Serial.begin(115200);

  PGM_VOID_P p = (void*) pgm_get_far_address(charSet);
  
  // put your setup code here, to run once:
  while(p != NULL)
  {
    Serial.print(pgm_read_byte(p), HEX);
    p = p + 1;
    
  }
}

void loop() {
  // put your main code here, to run repeatedly:
}

Feed to your favorite decompiler to get the AVR Assembly for your program. Can you reprogram your Arduino at runtime? Indeed you can. Watch this.

#include <avr/pgmspace.h>

const uint8_t MachineCode[44] PROGMEM = {
  0x25, 0x9A, 0x2D, 0x9A, 0x40, 0xE5, 0x5F, 0xEF,
  0x6F, 0xEF, 0x6A, 0x95, 0xF1, 0xF7, 0x5A, 0x95,
  0xE1, 0xF7, 0x4A, 0x95, 0xD1, 0xF7, 0x2D, 0x98,
  0x40, 0xE5, 0x5F, 0xEF, 0x6F, 0xEF, 0x6A, 0x95,
  0xF1, 0xF7, 0x5A, 0x95, 0xE1, 0xF7, 0x4A, 0x95,
  0xD1, 0xF7, 0xEB, 0xCF
};

const uint8_t *ptr = MachineCode;

void setup() {
  pinMode(2, INPUT);
}

void loop() {
  if (digitalRead(2))
  {
    //get address of the beginning of our code and call it
    asm(
      "lds r30, ptr   \n\t" // put our 
      "lds r31, ptr+1 \n\t" // get just past our ptr
      "lsr r30        \n\t" //convert byte address (data) to word address (flash)
      "icall          \n\t"
    );
  }
}

Now, that’s not the same as re-writing parts of PROGMEM, that’s actually just telling our program to run something else that lives in PROGMEM; basically we’ve uploaded two different programs and are pointing the “run this program” instruction to different parts of it. The moral of the story here is that PROGMEM is more than just a place to store strings, you can drop all kind of fun things into it and even though you can’t write it, you can do plenty with it nonetheless.