Skip to content

Validate mpy load #1972

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jun 27, 2019
Merged

Validate mpy load #1972

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
330517b
Validate portions of mpy load to detect corruption
tannewt Jun 25, 2019
77355cb
Update translations
tannewt Jun 25, 2019
2494bfd
Validate raw code and mpy read length
tannewt Jun 26, 2019
6d2072f
Fix tests
tannewt Jun 26, 2019
1f105e0
Update frozen libraries to save space
tannewt Jun 26, 2019
fe507db
Shrink pIRkey build down.
tannewt Jun 26, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion frozen/Adafruit_CircuitPython_BusDevice
Open in desktop
Submodule Adafruit_CircuitPython_BusDevice updated 1 files
+2 −18 adafruit_bus_device/i2c_device.py
2 changes: 1 addition & 1 deletion frozen/Adafruit_CircuitPython_CircuitPlayground
Open in desktop
Submodule Adafruit_CircuitPython_CircuitPlayground updated 37 files
+124 −12 docs/examples.rst
+0 −0 examples/circuitplayground_acceleration.py
+27 −0 examples/circuitplayground_acceleration_neopixels.py
+7 −0 examples/circuitplayground_button_a.py
+14 −0 examples/circuitplayground_button_b.py
+17 −0 examples/circuitplayground_buttons_1_neopixel.py
+17 −0 examples/circuitplayground_buttons_neopixels.py
+36 −0 examples/circuitplayground_ir_receive.py
+33 −0 examples/circuitplayground_ir_transmit.py
+9 −0 examples/circuitplayground_light.py
+26 −0 examples/circuitplayground_light_neopixels.py
+10 −0 examples/circuitplayground_light_plotter.py
+8 −0 examples/circuitplayground_neopixel_0_1.py
+10 −0 examples/circuitplayground_play_file.py
+14 −0 examples/circuitplayground_play_file_buttons.py
+6 −0 examples/circuitplayground_play_tone.py
+8 −0 examples/circuitplayground_play_tone_buttons.py
+5 −0 examples/circuitplayground_red_led.py
+10 −0 examples/circuitplayground_red_led_blinky.py
+8 −0 examples/circuitplayground_red_led_blnky_short.py
+0 −0 examples/circuitplayground_shake.py
+8 −0 examples/circuitplayground_slide_switch.py
+11 −0 examples/circuitplayground_slide_switch_red_led.py
+6 −0 examples/circuitplayground_slide_switch_red_led_short.py
+56 −0 examples/circuitplayground_sound_meter.py
+15 −0 examples/circuitplayground_tap_red_led.py
+0 −0 examples/circuitplayground_tapdetect.py
+0 −0 examples/circuitplayground_tapdetect_single_double.py
+32 −0 examples/circuitplayground_temperature_neopixels.py
+10 −0 examples/circuitplayground_temperture.py
+11 −0 examples/circuitplayground_temperture_plotter.py
+0 −0 examples/circuitplayground_tone.py
+31 −0 examples/circuitplayground_touch_pixel_fill_rainbow.py
+31 −0 examples/circuitplayground_touch_pixel_rainbow.py
+0 −0 examples/circuitplayground_touched.py
+ − examples/dip.wav
+ − examples/rise.wav
2 changes: 1 addition & 1 deletion frozen/Adafruit_CircuitPython_LIS3DH
Open in desktop
Submodule Adafruit_CircuitPython_LIS3DH updated 1 files
+2 −8 adafruit_lis3dh.py
2 changes: 1 addition & 1 deletion frozen/Adafruit_CircuitPython_Motor
Open in desktop
Submodule Adafruit_CircuitPython_Motor updated 1 files
+1 −1 README.rst
2 changes: 1 addition & 1 deletion frozen/Adafruit_CircuitPython_seesaw
Open in desktop
Submodule Adafruit_CircuitPython_seesaw updated 1 files
+1 −1 adafruit_seesaw/seesaw.py
2 changes: 1 addition & 1 deletion frozen/circuitpython-stage
Open in desktop
Submodule circuitpython-stage updated 2 files
+1 −0 pygamer/stage.py
+95 −0 pygamer/ugame.py
10 changes: 9 additions & 1 deletion locale/ID.po
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2019-05-13 17:34-0700\n"
"POT-Creation-Date: 2019-06-25 17:53-0700\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <[email protected]>\n"
Expand Down Expand Up @@ -460,6 +460,14 @@ msgstr ""
msgid "Command must be an int between 0 and 255"
msgstr ""

#: py/persistentcode.c
msgid "Corrupt .mpy file"
msgstr ""

#: py/emitglue.c
msgid "Corrupt raw code"
msgstr ""

#: ports/nrf/common-hal/bleio/UUID.c
#, c-format
msgid "Could not decode ble_uuid, err 0x%04x"
Expand Down
10 changes: 9 additions & 1 deletion locale/circuitpython.pot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2019-05-13 17:41-0700\n"
"POT-Creation-Date: 2019-06-25 17:53-0700\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <[email protected]>\n"
Expand Down Expand Up @@ -450,6 +450,14 @@ msgstr ""
msgid "Command must be an int between 0 and 255"
msgstr ""

#: py/persistentcode.c
msgid "Corrupt .mpy file"
msgstr ""

#: py/emitglue.c
msgid "Corrupt raw code"
msgstr ""

#: ports/nrf/common-hal/bleio/UUID.c
#, c-format
msgid "Could not decode ble_uuid, err 0x%04x"
Expand Down
10 changes: 9 additions & 1 deletion locale/de_DE.po
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ msgid ""
msgstr ""
"Project-Id-Version: \n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2019-05-13 17:34-0700\n"
"POT-Creation-Date: 2019-06-25 17:53-0700\n"
"PO-Revision-Date: 2018-07-27 11:55-0700\n"
"Last-Translator: Pascal Deneaux\n"
"Language-Team: Sebastian Plamauer, Pascal Deneaux\n"
Expand Down Expand Up @@ -454,6 +454,14 @@ msgstr ""
msgid "Command must be an int between 0 and 255"
msgstr "Der Befehl muss ein int zwischen 0 und 255 sein"

#: py/persistentcode.c
msgid "Corrupt .mpy file"
msgstr ""

#: py/emitglue.c
msgid "Corrupt raw code"
msgstr ""

#: ports/nrf/common-hal/bleio/UUID.c
#, c-format
msgid "Could not decode ble_uuid, err 0x%04x"
Expand Down
10 changes: 9 additions & 1 deletion locale/en_US.po
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ msgid ""
msgstr ""
"Project-Id-Version: \n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2019-05-13 17:34-0700\n"
"POT-Creation-Date: 2019-06-25 17:53-0700\n"
"PO-Revision-Date: 2018-07-27 11:55-0700\n"
"Last-Translator: \n"
"Language-Team: \n"
Expand Down Expand Up @@ -450,6 +450,14 @@ msgstr ""
msgid "Command must be an int between 0 and 255"
msgstr ""

#: py/persistentcode.c
msgid "Corrupt .mpy file"
msgstr ""

#: py/emitglue.c
msgid "Corrupt raw code"
msgstr ""

#: ports/nrf/common-hal/bleio/UUID.c
#, c-format
msgid "Could not decode ble_uuid, err 0x%04x"
Expand Down
10 changes: 9 additions & 1 deletion locale/en_x_pirate.po
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ msgid ""
msgstr ""
"Project-Id-Version: \n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2019-05-13 17:34-0700\n"
"POT-Creation-Date: 2019-06-25 17:53-0700\n"
"PO-Revision-Date: 2018-07-27 11:55-0700\n"
"Last-Translator: \n"
"Language-Team: @sommersoft, @MrCertainly\n"
Expand Down Expand Up @@ -454,6 +454,14 @@ msgstr ""
msgid "Command must be an int between 0 and 255"
msgstr ""

#: py/persistentcode.c
msgid "Corrupt .mpy file"
msgstr ""

#: py/emitglue.c
msgid "Corrupt raw code"
msgstr ""

#: ports/nrf/common-hal/bleio/UUID.c
#, c-format
msgid "Could not decode ble_uuid, err 0x%04x"
Expand Down
10 changes: 9 additions & 1 deletion locale/es.po
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ msgid ""
msgstr ""
"Project-Id-Version: \n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2019-05-13 17:34-0700\n"
"POT-Creation-Date: 2019-06-25 17:53-0700\n"
"PO-Revision-Date: 2018-08-24 22:56-0500\n"
"Last-Translator: \n"
"Language-Team: \n"
Expand Down Expand Up @@ -466,6 +466,14 @@ msgstr "La entrada en la columna debe ser digitalio.DigitalInOut"
msgid "Command must be an int between 0 and 255"
msgstr "Bytes debe estar entre 0 y 255."

#: py/persistentcode.c
msgid "Corrupt .mpy file"
msgstr ""

#: py/emitglue.c
msgid "Corrupt raw code"
msgstr ""

#: ports/nrf/common-hal/bleio/UUID.c
#, c-format
msgid "Could not decode ble_uuid, err 0x%04x"
Expand Down
10 changes: 9 additions & 1 deletion locale/fil.po
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ msgid ""
msgstr ""
"Project-Id-Version: \n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2019-05-13 17:34-0700\n"
"POT-Creation-Date: 2019-06-25 17:53-0700\n"
"PO-Revision-Date: 2018-12-20 22:15-0800\n"
"Last-Translator: Timothy <[email protected]>\n"
"Language-Team: fil\n"
Expand Down Expand Up @@ -459,6 +459,14 @@ msgstr ""
msgid "Command must be an int between 0 and 255"
msgstr "Sa gitna ng 0 o 255 dapat ang bytes."

#: py/persistentcode.c
msgid "Corrupt .mpy file"
msgstr ""

#: py/emitglue.c
msgid "Corrupt raw code"
msgstr ""

#: ports/nrf/common-hal/bleio/UUID.c
#, c-format
msgid "Could not decode ble_uuid, err 0x%04x"
Expand Down
10 changes: 9 additions & 1 deletion locale/fr.po
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ msgid ""
msgstr ""
"Project-Id-Version: 0.1\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2019-05-13 17:34-0700\n"
"POT-Creation-Date: 2019-06-25 17:53-0700\n"
"PO-Revision-Date: 2019-04-14 20:05+0100\n"
"Last-Translator: Pierrick Couturier <[email protected]>\n"
"Language-Team: fr\n"
Expand Down Expand Up @@ -465,6 +465,14 @@ msgstr "L'entrée 'Column' doit être un digitalio.DigitalInOut"
msgid "Command must be an int between 0 and 255"
msgstr "La commande doit être un entier entre 0 et 255"

#: py/persistentcode.c
msgid "Corrupt .mpy file"
msgstr ""

#: py/emitglue.c
msgid "Corrupt raw code"
msgstr ""

#: ports/nrf/common-hal/bleio/UUID.c
#, c-format
msgid "Could not decode ble_uuid, err 0x%04x"
Expand Down
10 changes: 9 additions & 1 deletion locale/it_IT.po
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2019-05-13 17:34-0700\n"
"POT-Creation-Date: 2019-06-25 17:53-0700\n"
"PO-Revision-Date: 2018-10-02 16:27+0200\n"
"Last-Translator: Enrico Paganin <[email protected]>\n"
"Language-Team: \n"
Expand Down Expand Up @@ -460,6 +460,14 @@ msgstr ""
msgid "Command must be an int between 0 and 255"
msgstr "I byte devono essere compresi tra 0 e 255"

#: py/persistentcode.c
msgid "Corrupt .mpy file"
msgstr ""

#: py/emitglue.c
msgid "Corrupt raw code"
msgstr ""

#: ports/nrf/common-hal/bleio/UUID.c
#, c-format
msgid "Could not decode ble_uuid, err 0x%04x"
Expand Down
10 changes: 9 additions & 1 deletion locale/pl.po
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ msgid ""
msgstr ""
"Project-Id-Version: \n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2019-05-13 17:34-0700\n"
"POT-Creation-Date: 2019-06-25 17:53-0700\n"
"PO-Revision-Date: 2019-03-19 18:37-0700\n"
"Last-Translator: Radomir Dopieralski <[email protected]>\n"
"Language-Team: pl\n"
Expand Down Expand Up @@ -453,6 +453,14 @@ msgstr "Kolumny muszą być typu digitalio.DigitalInOut"
msgid "Command must be an int between 0 and 255"
msgstr "Komenda musi być int pomiędzy 0 a 255"

#: py/persistentcode.c
msgid "Corrupt .mpy file"
msgstr ""

#: py/emitglue.c
msgid "Corrupt raw code"
msgstr ""

#: ports/nrf/common-hal/bleio/UUID.c
#, c-format
msgid "Could not decode ble_uuid, err 0x%04x"
Expand Down
10 changes: 9 additions & 1 deletion locale/pt_BR.po
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2019-05-13 17:34-0700\n"
"POT-Creation-Date: 2019-06-25 17:53-0700\n"
"PO-Revision-Date: 2018-10-02 21:14-0000\n"
"Last-Translator: \n"
"Language-Team: \n"
Expand Down Expand Up @@ -456,6 +456,14 @@ msgstr ""
msgid "Command must be an int between 0 and 255"
msgstr "Os bytes devem estar entre 0 e 255."

#: py/persistentcode.c
msgid "Corrupt .mpy file"
msgstr ""

#: py/emitglue.c
msgid "Corrupt raw code"
msgstr ""

#: ports/nrf/common-hal/bleio/UUID.c
#, c-format
msgid "Could not decode ble_uuid, err 0x%04x"
Expand Down
10 changes: 9 additions & 1 deletion locale/zh_Latn_pinyin.po
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ msgid ""
msgstr ""
"Project-Id-Version: circuitpython-cn\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2019-05-13 17:34-0700\n"
"POT-Creation-Date: 2019-06-25 17:53-0700\n"
"PO-Revision-Date: 2019-04-13 10:10-0700\n"
"Last-Translator: hexthat\n"
"Language-Team: Chinese Hanyu Pinyin\n"
Expand Down Expand Up @@ -454,6 +454,14 @@ msgstr "Liè tiáomù bìxū shì digitalio.DigitalInOut"
msgid "Command must be an int between 0 and 255"
msgstr "Mìnglìng bìxū shì 0 dào 255 zhī jiān de int"

#: py/persistentcode.c
msgid "Corrupt .mpy file"
msgstr ""

#: py/emitglue.c
msgid "Corrupt raw code"
msgstr ""

#: ports/nrf/common-hal/bleio/UUID.c
#, c-format
msgid "Could not decode ble_uuid, err 0x%04x"
Expand Down
3 changes: 0 additions & 3 deletions ports/atmel-samd/boards/pirkey_m0/pins.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,5 @@ STATIC const mp_rom_map_elem_t board_global_dict_table[] = {

{ MP_ROM_QSTR(MP_QSTR_APA102_MOSI), MP_ROM_PTR(&pin_PA00) },
{ MP_ROM_QSTR(MP_QSTR_APA102_SCK), MP_ROM_PTR(&pin_PA01) },
{ MP_ROM_QSTR(MP_QSTR_I2C), MP_ROM_PTR(&board_i2c_obj) },
{ MP_ROM_QSTR(MP_QSTR_SPI), MP_ROM_PTR(&board_spi_obj) },
{ MP_ROM_QSTR(MP_QSTR_UART), MP_ROM_PTR(&board_uart_obj) },
};
MP_DEFINE_CONST_DICT(board_module_globals, board_global_dict_table);
7 changes: 4 additions & 3 deletions py/emitglue.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,11 +142,12 @@ mp_obj_t mp_make_function_from_raw_code(const mp_raw_code_t *rc, mp_obj_t def_ar
fun = mp_obj_new_fun_asm(rc->n_pos_args, rc->data.u_native.fun_data, rc->data.u_native.type_sig);
break;
#endif
default:
// rc->kind should always be set and BYTECODE is the only remaining case
assert(rc->kind == MP_CODE_BYTECODE);
case MP_CODE_BYTECODE:
fun = mp_obj_new_fun_bc(def_args, def_kw_args, rc->data.u_byte.bytecode, rc->data.u_byte.const_table);
break;
default:
// All other kinds are invalid.
mp_raise_RuntimeError(translate("Corrupt raw code"));
}

// check for generator functions and if so wrap in generator object
Expand Down
26 changes: 21 additions & 5 deletions py/persistentcode.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,20 +102,35 @@ STATIC void extract_prelude(const byte **ip, const byte **ip2, bytecode_prelude_

#include "py/parsenum.h"

STATIC void raise_corrupt_mpy(void) {
mp_raise_RuntimeError(translate("Corrupt .mpy file"));
}

STATIC int read_byte(mp_reader_t *reader) {
return reader->readbyte(reader->data);
mp_uint_t b = reader->readbyte(reader->data);
if (b == MP_READER_EOF) {
raise_corrupt_mpy();
}
return b;
}

STATIC void read_bytes(mp_reader_t *reader, byte *buf, size_t len) {
while (len-- > 0) {
*buf++ = reader->readbyte(reader->data);
mp_uint_t b =reader->readbyte(reader->data);
if (b == MP_READER_EOF) {
raise_corrupt_mpy();
}
*buf++ = b;
}
}

STATIC size_t read_uint(mp_reader_t *reader) {
size_t unum = 0;
for (;;) {
byte b = reader->readbyte(reader->data);
mp_uint_t b = reader->readbyte(reader->data);
if (b == MP_READER_EOF) {
raise_corrupt_mpy();
}
unum = (unum << 7) | (b & 0x7f);
if ((b & 0x80) == 0) {
break;
Expand Down Expand Up @@ -145,11 +160,12 @@ STATIC mp_obj_t load_obj(mp_reader_t *reader) {
return mp_obj_new_str_from_vstr(obj_type == 's' ? &mp_type_str : &mp_type_bytes, &vstr);
} else if (obj_type == 'i') {
return mp_parse_num_integer(vstr.buf, vstr.len, 10, NULL);
} else {
assert(obj_type == 'f' || obj_type == 'c');
} else if (obj_type == 'f' || obj_type == 'c') {
return mp_parse_num_decimal(vstr.buf, vstr.len, obj_type == 'c', false, NULL);
}
}
raise_corrupt_mpy();
return MP_OBJ_FROM_PTR(&mp_const_none_obj);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The return value is a size_t, not an mp_obj_t. If the compiler is forcing you to return something, even though it will never get executed, just return 0, I'd say.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never mind, confused by diff listing with missing lines.

}

STATIC void load_bytecode_qstrs(mp_reader_t *reader, byte *ip, byte *ip_top) {
Expand Down
4 changes: 2 additions & 2 deletions tests/import/mpy_invalid.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,8 +59,8 @@ def open(self, path, mode):
mod = 'mod%u' % i
try:
__import__(mod)
except ValueError as er:
print(mod, 'ValueError', er)
except Exception as e:
print(mod, type(e).__name__, e)

# unmount and undo path addition
uos.umount('/userfs')
Expand Down
Loading