weixin_39694174
weixin_39694174
2020-12-01 23:47

Allow null padding bytes in GBL files (fixes #525)

Hue GBL firmware images contain unexpected null bytes as padding and thus fail validation. This fix relaxes the constraint.

Below are the validation results after this patch for every known OTA file:


Not an OTA image: ConnectedLamp-Atmel_0104_5.130.1.30000_0012.sbl-ota, Data is too short for SubElement
Not an OTA image: LivCol_0103_5.127.1.26581_0012.sbl-ota, Data is too short for SubElement
Not an OTA image: LivingColors-Target_0108_5.130.1.30000_0012.sbl-ota, Data is too short for SubElement
Not an OTA image: ModuLum-ATmega_010B_5.130.1.30000_0012.sbl-ota, Data is too short for SubElement
Not an OTA image: Sensor-ATmega_6.1.1.27575_0012.sbl-ota, Data is too short for SubElement
Not an OTA image: Switch-ATmega_6.1.1.28573_0012.sbl-ota, Data is too short for SubElement
Not an OTA image: TI_0100_5.127.1.26581_0012.sbl-ota, Data is too short for SubElement
Not an OTA image: WhiteLamp-Atmel-Target_0105_5.130.1.30000_0012.sbl-ota, Data is too short for SubElement


ValidationResult.INVALID: ikea-firmwares/bad/10005778-10.1-TRADFRI-onoff-shortcut-control-2.2.010.ota.ota.signed
ValidationResult.INVALID: ikea-firmwares/bad/10043101-3.1-TRADFRI-dimmer-2.1.024.ota.ota.signed
ValidationResult.INVALID: ikea-firmwares/bad/10005777-6.1-TRADFRI-control-outlet-2.0.024.ota.ota.signed
ValidationResult.INVALID: ikea-firmwares/bad/10046695-1.1-TRADFRI-light-unified-w-2.3.050.ota.ota.signed
ValidationResult.INVALID: ikea-firmwares/bad/10040611-3.2-TRADFRI-sy5882-unified-2.3.050.ota.ota.signed
ValidationResult.INVALID: ikea-firmwares/bad/10047227-1.2-TRADFRI-cv-cct-unified-2.3.050.ota.ota.signed
ValidationResult.INVALID: ikea-firmwares/bad/10038562-2.1-TRADFRI-sy5882-bulb-ws-2.0.029.ota.ota.signed
ValidationResult.INVALID: ikea-firmwares/bad/191100-4.1-TRADFRI-sy5882-driver-ws-2.0.029.ota.ota.signed


ValidationResult.UNKNOWN: 100B-010C-01001A02-ConfLight-Lamps_0012.zigbee
ValidationResult.UNKNOWN: 100B-010E-01001904-ConfLight-ModuLum_0012.zigbee
ValidationResult.UNKNOWN: 100B-010F-01000A02-ConfLight-LedStrips_0012.zigbee
ValidationResult.UNKNOWN: 1135-0000-201000A0-FLS-PP3_RGBW.zigbee
ValidationResult.UNKNOWN: A19_RGBW_IMG0019_00102428-encrypted.ota
ValidationResult.UNKNOWN: A19_TW_10_year_IMG000D_00102428-encrypted.ota
ValidationResult.UNKNOWN: A19_W_10_year_IMG000C_00102428-encrypted.ota
ValidationResult.UNKNOWN: A60_DIM_Z3_IM003D_00103101-encrypted_11_20_2018_Tue_122925_01_withoutMF.ota
ValidationResult.UNKNOWN: A60_TW_Z3_IM003C_00103101-encrypted_11_20_2018_Tue_103138_93_withoutMF.ota
ValidationResult.UNKNOWN: B40_DIM_Z3_IM0034_00103101-encrypted_11_26_2018_Mon_174522_20_withoutMF.ota
ValidationResult.UNKNOWN: B40_TW_Z3_IM0033_00103101-encrypted_11_23_2018_Fri_160706_13_withoutMF.ota
ValidationResult.UNKNOWN: BR30_RGBW_IMG001B_00102428-encrypted.ota
ValidationResult.UNKNOWN: BR30_TW_IMG001A_00102428-encrypted.ota
ValidationResult.UNKNOWN: BR30_W_10_year_IMG000F_00102428-encrypted.ota
ValidationResult.UNKNOWN: CLA60_RGBW_Z3_IM0011_00103101-encrypted_11_27_2018_Tue_133608_15_withoutMF.ota
ValidationResult.UNKNOWN: Conv_Under_Cabinet_TW_IMG0021_00102428-encrypted.ota
ValidationResult.UNKNOWN: Downlight_TW_HCL_IM0065_00103201-encrypted_09_20_2019_Fri_142050_70_withoutMF.ota
ValidationResult.UNKNOWN: Edge_Lit_Under_Cabinet_IMG0023_00102411-encrypted.ota
ValidationResult.UNKNOWN: FLEX_Outdoor_RGBW_IMG001F_00102428-encrypted.ota
ValidationResult.UNKNOWN: FLEX_RGBW_IMG001E_00102428-encrypted.ota
ValidationResult.UNKNOWN: Flex_RGBW_Z3_IM002A_00103101-encrypted_11_27_2018_Tue_134318_76_withoutMF.ota
ValidationResult.UNKNOWN: Flushmount_TW_IMG0022_00102428-encrypted.ota
ValidationResult.UNKNOWN: Gardenpole_Mini_RGBW_Z3_IM0040_00103103-encrypted_02_27_2019_Wed_151557_92_withoutMF.ota
ValidationResult.UNKNOWN: Gardenpole_RGBW_Z3_IM003B_00103103-encrypted_02_27_2019_Wed_150725_31_withoutMF.ota
ValidationResult.UNKNOWN: Outdoor_Accent_Light_RGB_IMG0020_00102428-encrypted.ota
ValidationResult.UNKNOWN: Outdoor_FLEX_RGBW_Z3_IM005C_00103101-encrypted_11_27_2018_Tue_135739_87_withoutMF.ota
ValidationResult.UNKNOWN: Panel_TW_HCL_IM0063_00103201-encrypted_09_18_2019_Wed_113705_07_withoutMF.ota
ValidationResult.UNKNOWN: Panel_TW_Z3_IM005A_00103101-encrypted_11_23_2018_Fri_161331_81_withoutMF.ota
ValidationResult.UNKNOWN: PAR16_DIM_Z3_IM0031_00103101-encrypted_11_26_2018_Mon_175052_32_withoutMF.ota
ValidationResult.UNKNOWN: PAR16_RGBW_Z3_IM0030_00103101-encrypted_11_27_2018_Tue_140612_79_withoutMF.ota
ValidationResult.UNKNOWN: PAR16_TW_Z3_IM002E_00103101-encrypted_11_23_2018_Fri_162418_58_withoutMF.ota
ValidationResult.UNKNOWN: PAR38_W_10_year_IMG0010_00102428-encrypted.ota
ValidationResult.UNKNOWN: Plug_Z3_IM002D_00103101-encrypted_12_07_2018_Fri_103650_94_withoutMF.ota
ValidationResult.UNKNOWN: RT_RGBW_IMG001D_00102428-encrypted.ota
ValidationResult.UNKNOWN: RT_TW_IMG001C_00102428-encrypted.ota
ValidationResult.UNKNOWN: Tibea_TW_Z3_IM002C_00103101-encrypted_11_23_2018_Fri_163423_97_withoutMF.ota
ValidationResult.UNKNOWN: Undercabinet_TW_Z3_IM0046_00103101-encrypted_11_20_2018_Tue_101550_96_withoutMF.ota
ValidationResult.UNKNOWN: ZLL_HS_4f_GJ_Release_10.03.32.02.zigbee
ValidationResult.UNKNOWN: ZLL_MK_0x01020509_CLA60_TW.ota
ValidationResult.UNKNOWN: ZLL_MK_0x01020510_CEILING_TW_OSRAM.ota
ValidationResult.UNKNOWN: ZLL_MK_0x01020510_CLA60_RGBW_OSRAM.ota
ValidationResult.UNKNOWN: ZLL_MK_0x01020510_CLA60_TW_OSRAM.ota
ValidationResult.UNKNOWN: ZLL_MK_0x01020510_CLA60_W_CLEAR.ota
ValidationResult.UNKNOWN: ZLL_MK_0x01020510_CLASSIC_A60_RGBW.ota
ValidationResult.UNKNOWN: ZLL_MK_0x01020510_CLASSIC_B40_TW.ota
ValidationResult.UNKNOWN: ZLL_MK_0x01020510_FLOOD_LIGHT_RGBW_OSRAM.ota
ValidationResult.UNKNOWN: ZLL_MK_0x01020510_GARDENPOLE_MINI_RGBW_OSRAM.ota
ValidationResult.UNKNOWN: ZLL_MK_0x01020510_GARDENPOLE_RGBW_LIGHTIFY.ota
ValidationResult.UNKNOWN: ZLL_MK_0x01020510_GARDENSPOT_RGB.ota
ValidationResult.UNKNOWN: ZLL_MK_0x01020510_GARDENSPOT_W.ota
ValidationResult.UNKNOWN: ZLL_MK_0x01020510_LIGHTIFY_INDOOR_FLEX.ota
ValidationResult.UNKNOWN: ZLL_MK_0x01020510_LIGHTIFY_OUTDOOR_FLEX.ota
ValidationResult.UNKNOWN: ZLL_MK_0x01020510_MR16_TW_OSRAM.ota
ValidationResult.UNKNOWN: ZLL_MK_0x01020510_OUTDOOR_LANTERN_B50_RGBW_OSRAM.ota
ValidationResult.UNKNOWN: ZLL_MK_0x01020510_OUTDOOR_LANTERN_B90_RGBW_OSRAM.ota
ValidationResult.UNKNOWN: ZLL_MK_0x01020510_OUTDOOR_LANTERN_W_RGBW_OSRAM.ota
ValidationResult.UNKNOWN: ZLL_MK_0x01020510_PANEL_RGBW_OSRAM.ota
ValidationResult.UNKNOWN: ZLL_MK_0x01020510_PAR16_50_TW.ota
ValidationResult.UNKNOWN: ZLL_MK_0x01020510_Par16Rgbw.ota
ValidationResult.UNKNOWN: ZLL_MK_0x01020510_Surface_Light_TW.ota
ValidationResult.UNKNOWN: ZLL_MK_0x01020510_Surface_Light_W.ota
ValidationResult.UNKNOWN: ZLL_Plug01_OnOff_MK_0x01020509.ota
ValidationResult.UNKNOWN: ZLL_SubstiTube_W_MK_0x01020509.ota
ValidationResult.UNKNOWN: ZLL_WS_4f_J_Release_10.03.32.02.zigbee


ValidationResult.VALID: 100B-0110-01001600-ConfLight-Lamps-EFR32MG13.zigbee
ValidationResult.VALID: 100B-0111-01001600-ConfLight-ModuLum-EFR32MG13.zigbee
ValidationResult.VALID: 100B-0112-01001500-ConfLightBLE-Lamps-EFR32MG13.zigbee
ValidationResult.VALID: 100B-0114-01000E04-ConfLightBLE-Lamps-EFR32MG21.zigbee
ValidationResult.VALID: 100B-0115-01000800-SmartPlug-EFR32MG13.zigbee
ValidationResult.VALID: 100B-0116-02001E00-Switch-EFR32MG13.zigbee
ValidationResult.VALID: 1542704530003_RDS2017007_E11-N1EA_V0.0.49_20181026_release.ota
ValidationResult.VALID: 1554860326863_RDS2018021_E1G-G8E_V10_20190405_release.ota
ValidationResult.VALID: 1555679540244_RDS2017009_E11_U2E_V42_20190418_release.ota
ValidationResult.VALID: 1600167449-si32_zg_uart_connect_sleep_ZS5_ty_OTA_1.1.5.bin
ValidationResult.VALID: ikea-firmwares/bad/10035514-2.1-TRADFRI-bulb-ws-2.3.050.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/bad/10035515-TRADFRI-bulb-cws-1.3.013.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/bad/10035534-2.1-TRADFRI-bulb-ws-gu10-2.3.050.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/bad/10037585-5.1-TRADFRI-connected-blind-2.2.009.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/bad/10037603-3.1-TRADFRI-signal-repeater-2.2.005.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/bad/10039874-1.0-TRADFRI-motion-sensor-2-2.0.022.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/bad/159495-TRADFRI-transformer-1.2.245.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/bad/159695-2.1-TRADFRI-bulb-ws-1000lm-2.3.050.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/bad/159696-TRADFRI-bulb-w-1000lm-1.2.214.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/bad/159697-TRADFRI-driver-hp-1.2.224.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/bad/159698-TRADFRI-driver-lp-1.2.224.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/bad/159699-5.1-TRADFRI-remote-control-2.3.014.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/bad/159700-TRADFRI-motion-sensor-1.2.214.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/bad/159701-2.1-TRADFRI-wireless-dimmer-2.3.028.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/bad/190579-ncp572b444.ebl.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/good/10005777-4.1-TRADFRI-control-outlet-2.0.024.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/good/10005778-10.1-TRADFRI-onoff-shortcut-control-2.2.010.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/good/10035514-2.1-TRADFRI-bulb-ws-2.3.050.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/good/10035515-TRADFRI-bulb-cws-1.3.013.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/good/10035534-2.1-TRADFRI-bulb-ws-gu10-2.3.050.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/good/10037585-5.1-TRADFRI-connected-blind-2.2.009.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/good/10037603-3.1-TRADFRI-signal-repeater-2.2.005.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/good/10038562-2.1-TRADFRI-sy5882-bulb-ws-2.0.029.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/good/10039874-1.0-TRADFRI-motion-sensor-2-2.0.022.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/good/10040611-3.2-TRADFRI-sy5882-unified-2.3.050.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/good/10043101-3.1-TRADFRI-dimmer-2.1.024.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/good/10046695-1.1-TRADFRI-light-unified-w-2.3.050.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/good/10047227-1.2-TRADFRI-cv-cct-unified-2.3.050.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/good/159495-TRADFRI-transformer-1.2.245.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/good/159695-2.1-TRADFRI-bulb-ws-1000lm-2.3.050.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/good/159696-TRADFRI-bulb-w-1000lm-2.3.023.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/good/159697-TRADFRI-driver-hp-1.2.224.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/good/159698-TRADFRI-driver-lp-1.2.224.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/good/159699-5.1-TRADFRI-remote-control-2.3.014.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/good/159700-TRADFRI-motion-sensor-1.2.214.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/good/159701-2.1-TRADFRI-wireless-dimmer-2.3.028.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/good/190579-ncp572b444.ebl.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/good/191100-4.1-TRADFRI-sy5882-driver-ws-2.0.029.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/last-good/10005777-6.1-TRADFRI-control-outlet-2.0.024.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/last-good/10005778-10.1-TRADFRI-onoff-shortcut-control-2.2.010.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/last-good/10035514-2.1-TRADFRI-bulb-ws-2.3.007.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/last-good/10035515-TRADFRI-bulb-cws-1.3.009.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/last-good/100355342-2.1-TRADFRI-bulb-ws-gu10-2.3.007.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/last-good/10037585-5.1-TRADFRI-connected-blind-2.2.009.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/last-good/10037603-3.1-TRADFRI-signal-repeater-2.2.005.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/last-good/10038562-2.1-TRADFRI-sy5882-bulb-ws-2.0.023.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/last-good/10039874-1.0-TRADFRI-motion-sensor-2-2.0.022.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/last-good/10040611-3.2-TRADFRI-sy5882-unified-2.1.022.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/last-good/10043101-3.1-TRADFRI-dimmer-2.1.024.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/last-good/10046695-1.1-TRADFRI-light-unified-w-2.1.022.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/last-good/10047227-1.2-TRADFRI-cv-cct-unified-2.1.022.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/last-good/159495-TRADFRI-transformer-1.2.245.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/last-good/159695-2.1-TRADFRI-bulb-ws-1000lm-2.3.007.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/last-good/159696-TRADFRI-bulb-w-1000lm-1.2.214.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/last-good/159697-TRADFRI-driver-hp-1.2.217.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/last-good/159698-TRADFRI-driver-lp-1.2.217.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/last-good/159699-5.1-TRADFRI-remote-control-2.3.014.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/last-good/159700-TRADFRI-motion-sensor-1.2.214.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/last-good/159701-2.1-TRADFRI-wireless-dimmer-2.3.028.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/last-good/190579-ncp572b444.ebl.ota.ota.signed
ValidationResult.VALID: ikea-firmwares/last-good/191100-4.1-TRADFRI-sy5882-driver-ws-2.0.022.ota.ota.signed
ValidationResult.VALID: RDL2016091_1_E11-G13_V0.0.9_20170921_release.ota
ValidationResult.VALID: RDS2014011_Z01-A19_V0.0.46_20171028_release.ota
ValidationResult.VALID: RDS2017028_E1C-NB6_V0.0.22_20180314.ota
ValidationResult.VALID: Superman_v3_08_ProdKey_3080.ota

The Hue .sbl-ota files are not even valid Zigbee OTA files and are not directly related to this bugfix but I will try to reverse engineer the format (if there is one). I suspect they contain just a Zigbee OTA header and then the image data right after, without an enclosing subelement.

该提问来源于开源项目:zigpy/zigpy

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享
  • 邀请回答

6条回答

  • weixin_39629969 weixin_39629969 5月前

    Coverage Status

    Coverage increased (+2.0e-05%) to 99.869% when pulling b4b0d912c10e9bb94cb48e460d72b3d7f8a120a6 on puddly:bugfix/hue-ota-parsing into 96c8d6a0c5ccd7f9b0e3654d553d736b49bc1967 on zigpy:dev.

    点赞 评论 复制链接分享
  • weixin_39629969 weixin_39629969 5月前

    Coverage Status

    Coverage increased (+2.0e-05%) to 99.869% when pulling b4b0d912c10e9bb94cb48e460d72b3d7f8a120a6 on puddly:bugfix/hue-ota-parsing into 96c8d6a0c5ccd7f9b0e3654d553d736b49bc1967 on zigpy:dev.

    点赞 评论 复制链接分享
  • weixin_39694174 weixin_39694174 5月前

    The OTA file is still sent to the device unchanged, it's just that the OTA files are completely parsed to extract the header and the enclosed firmware image so it may possibly be validated (in order to prevent OTA loops where the device continuously upgrades to a defective image and reboots after this fails).

    Encrypted images exist and they're handled fine. The problem is that those sbl-ota files aren't actually valid Zigbee OTA files. They contain a header but they are missing the subsequent two byte subelement id, the four byte subelement length, and then the subelement payload that would typically by a firmware image (identified by the subelement id).

    Instead, I believe the firmware image is concatenated right to the end of the header, which will require a special parser for these specific images to extract the header and superficially validate the rest. The IKEA OTA container format also requires special handling but in a different way than it is when downloaded from IKEA's servers. That code I think should be reformatted and a similar "extractor" (or whatever you want to call it) be written for these weird Hue images.

    点赞 评论 复制链接分享
  • weixin_39830688 weixin_39830688 5月前

    Maybe should change the approach and send the file starting with the ota header till end of file? If it doesn't fit the ota format as per specs?

    点赞 评论 复制链接分享
  • weixin_39694174 weixin_39694174 5月前

    Yeah. These are the only OTA files that I've come across that actually deviate from the Zigbee spec so I don't think it'll be that much work to "handle" their format since it's just splitting off the header and ensuring the next few bytes are as expected.

    Maybe we should replace usage of OTAImage with an object that just contains an OTAImageHeader and the raw image bytes that will be sent off? We can still try to parse the Zigbee OTA image and validate the embedded firmware before this object is created (if possible, to reject corrupted images), but relaxing the parsing should reduce needless serialization during the OTA process.

    This should also "fix" the IKEA container format being explicitly parsed by the IKEA HTTP handler while being implicitly parsed by the FileStore handler's searching for the Zigbee OTA header in the first 512 bytes of every file.

    点赞 评论 复制链接分享
  • weixin_39830688 weixin_39830688 5月前

    That would work. Per se we're only interested in the ota header.

    点赞 评论 复制链接分享

相关推荐