Audio codecs locking up RP2040

[LowderPlay]

Hi! Thanks for such an amazing library!

I am using a custom RP2040 PCB with an SD card and I2S microphone (INMP441), I am planning to record audio snippets, encode them in a lossy format and store on the SD card.

Currently, I only managed to get WAVEncoder working and the audio sounds great. However, when I'm trying to use other codecs such as OpusAudioEncoder and MP3EncoderLAME, the RP2040 freezes without starting the actual recording. My guess is that encoding is taking up the entire RAM and the CPU becomes unresponsive.

#include <Arduino.h>
#include <SPI.h>
#include <SD.h>
#include <Wire.h>

const int RP_CLK_GPIO = 2;
const int RP_CMD_GPIO = 7;
const int RP_DAT0_GPIO = 3;
const int RECORD = 23;

#include "AudioTools.h"
#include "AudioTools/AudioCodecs/CodecMP3LAME.h"

File outputFile;
EncodedAudioStream out(&outputFile, new MP3EncoderLAME());
I2SStream i2sStream;    // Access I2S as stream
StreamCopy copier(out, i2sStream);  // copies data

int start = 0;

void setup() {
  Serial.begin(115200);

  while (!Serial) {
    delay(1);  // wait for serial port to connect. Needed for native USB port only
  }

  pinMode(RECORD, OUTPUT);

  AudioLogger::instance().begin(Serial, AudioLogger::Info);
  if(!SD.begin(RP_CLK_GPIO, RP_CMD_GPIO, RP_DAT0_GPIO)) {
    Serial.println("SD init failed!");
  }
  SD.remove("/test.mp3");
  outputFile = SD.open("/test.mp3", FILE_WRITE);
  if (!outputFile) {
    Serial.println("File open failed!");
    return;
  }

  
  Serial.println("starting I2S...");
  auto config = i2sStream.defaultConfig(RX_MODE);
  config.i2s_format = I2S_PHILIPS_FORMAT; // if quality is bad change to I2S_LSB_FORMAT https://github.com/pschatzmann/arduino-audio-tools/issues/23
  config.sample_rate = 8000;
  config.channels = 2;
  config.bits_per_sample = 16;
  config.pin_bck = 18;
  config.pin_ws = 19;
  config.pin_data = 20;

  i2sStream.begin(config);
  Serial.println("I2S started");
  out.begin(config);

  Serial.println("Test started...");
  start = millis();
  digitalWrite(RECORD, LOW);
}

void loop() { 
  if ((millis() - start) < 10000) {
    // record to wav file
    copier.copy();
  } else {
    // close file when done
    if (outputFile) {
      digitalWrite(RECORD, HIGH);
      outputFile.flush();
      Serial.print("File has ");
      Serial.print(outputFile.size());
      Serial.println(" bytes");
      outputFile.close();
    }
  }
}

This code produces this output:

[I] I2SRP2040.h : 51 - bool audio_tools::I2SDriverRP2040::begin(audio_tools::I2SConfigStd)
[I] AudioTypes.h : 124 -  sample_rate: 8000 / channels: 2 / bits_per_sample: 16
[I] I2SConfigStd.h : 75 - rx/tx mode: RX_MODE
[I] I2SConfigStd.h : 77 - is_master: Master
[I] I2SConfigStd.h : 78 - sample rate: 8000
[I] I2SConfigStd.h : 79 - bits per sample: 16
[I] I2SConfigStd.h : 80 - number of channels: 2
[I] I2SConfigStd.h : 81 - i2s_format: I2S_PHILIPS_FORMAT
[I] I2SConfigStd.h : 82 - buffer_count:6
[I] I2SConfigStd.h : 83 - buffer_size:512
[I] I2SConfigStd.h : 89 - pin_bck: 18
[I] I2SConfigStd.h : 91 - pin_ws: 19
[I] I2SConfigStd.h : 93 - pin_data: 20
I2S started
[I] AudioTypes.h : 124 - in: sample_rate: 8000 / channels: 2 / bits_per_sample: 16
[I] AudioTypes.h : 124 - out: sample_rate: 8000 / channels: 2 / bits_per_sample: 16
[I] CodecMP3LAME.h : 140 - LibLAME channels: 2
[I] CodecMP3LAME.h : 141 - LibLAME sample_rate: 8000
[I] CodecMP3LAME.h : 142 - LibLAME bits_per_sample: 16
[I] CodecMP3LAME.h : 143 - LibLAME quality: 7

Almost the same thing happens with opus:

OpusAudioEncoder enc;
EncodedAudioStream out(&outputFile, new OggContainerEncoder(enc));
...
  auto &encoderConfig = enc.config();
  encoderConfig.sample_rate = 8000;
  encoderConfig.application = OPUS_APPLICATION_AUDIO;
  encoderConfig.vbr = 1;
  encoderConfig.max_bandwidth = OPUS_BANDWIDTH_NARROWBAND;
  out.begin(encoderConfig);

I2S started
[I] AudioTypes.h : 124 - in: sample_rate: 8000 / channels: 2 / bits_per_sample: 16
[I] AudioTypes.h : 124 - out: sample_rate: 8000 / channels: 2 / bits_per_sample: 16
[I] AudioTypes.h : 124 -  sample_rate: 8000 / channels: 2 / bits_per_sample: 16
Test started...
[I] Buffers.h : 391 - resize: 1024

After the lockup, RP2040 is not flashable with picotool anymore (until rebooting). Also, the RECORD LED stays on forever (it turns off when using the WAVEncoder, as it should).

Any help would be appreciated! Cheers

[pschatzmann]

As described in the Readme, MP3 and AAC encoding requires a lot of RAM. You might be able to do this if you Microcontroller supports PSRAM, but there is no chance with the limited RAM that the RP2040 has!

Use a Codec that is more lightweight...

ps. The ESP32 generally has more RAM and flash memory than the RP2040. ESP32 models can offer up to 520KB of SRAM and up to 16MB of flash, while the RP2040 has 264KB of RAM and supports up to 16MB of off-chip flash.

[LowderPlay]

The same thing applies to libopus? I've seen other projects (without this library) running on rp2040 using opus encoding.

[pschatzmann]

I was only using this for decoding on a RP2040, so I don't know...