weixin_39613561
weixin_39613561
2020-11-22 11:02

Internal Server Error (500) with mediaItems:batchGet

I'm on f95c72cedd8f0a74b1501ce248f062ad70cd4aaa.

After downloading a few hundred files I get a crash with the following log:


01-17 11:53:08 gphotos.utils DEBUG    retry 0 failed: <bound method method.do_execute of object at>>
01-17 11:53:08 gphotos.utils DEBUG    retry 1 failed: <bound method method.do_execute of object at>>
01-17 11:53:09 gphotos.utils DEBUG    retry 2 failed: <bound method method.do_execute of object at>>
01-17 11:53:10 gphotos.utils DEBUG    retry 3 failed: <bound method method.do_execute of object at>>
01-17 11:53:11 gphotos.utils DEBUG    retry 4 failed: <bound method method.do_execute of object at>>
01-17 11:53:11 gphotos.Photos ERROR    failure in batch get of ['ACRh3gqHl1JT7bI0JBwAZS7tb8X7lyAKstKZ-qulC_p_iUTs593VZRFqVUuRLB6zPYxUAkU5AXCtTPTk-6o3G1Gi_GfIfXo1xg', 'ACRh3grwr4QLizR3okwzE6o3eMIflopt2WM1fic_PqGqn3SXBpQIF3yHYjHTGLNBm1m_2CL-rPUE_UnaVFh6P19x_MrPRxQXdg', 'ACRh3goT1pZL18jyksVWlwxwR78F0LiufYrmASx8DhMNuH3D8vdYmKtqiLsLdW14-S-TW4RUJM-guwM2Ptbddlrra_VoTuZJ5w', 'ACRh3gpoWhwl6nPKM-x3NGHdCozYeK2-kvhre8FSD2BvYtaMjFARLyUz99uyL1C53cPiEi_l9Oh6cqeD1BmZLyhYPJptaHlWQg', 'ACRh3gqXInOYSiiOc4piYTCONw5TObVmRFvlkDOSZYOgWpjdevMUT4Xc5KJe9QWAgH0zgfFvraZEbKqqJ1unVpOwaAqSRF4ALw', 'ACRh3gqFt0d1v7B89jAMJfsjmD3rQ1gmWdkSD2iRKaxuOxlWnP1tPO7fWMzC9L4ExIs_7dCdcJ-pJ2QrtMYkFZuZiSNDhrxXLg', 'ACRh3gpQFqmjoVvqho9T6JTQjvRCwR9De8f13e6sr0FYHJd2V_AjmR2UKoCpyH5Aau5dw0hse6dH3YF297Kk3EVMLtnlJXdxpA', 'ACRh3goOqk8j-Cd1YlFOVsq1kHvTgx2ArkHlaGqNGCKsEFWnfrzVEcDN9XqerKXSWeeTx-2fWkisgxMb4-D0ixtWK8qqKzS1Vg', 'ACRh3grmrooSV-dHeAigKCCipInH7mjRyeOrJNjUDoR8X1deqlFJXMRfJyrj_17ZbZBT1dfYQ1ciLjqeW3LVqcJdzFkQPygfAw', 'ACRh3gozoapWXPx3rkAl3NIiI_Qu9WYSCTHwnO2eRwAMnZFIsqzOmJzShM405dFhvRGxxexZifQx0KInMP2P7tE4V4wn6zQ28Q', 'ACRh3goTb8UG0tbhs165u_ASH-mCU7rZtT0vetyX5_kEzOwzacEAh2Fdq5xX03_cFo8MJoBNcLApG7MxYfa1FmjJRUxgRxDCfg', 'ACRh3gr99yaFQivyGCfLGXQnyX1UXgJxbTtHKa9BmSYcYeKjI3qOSCq1hCzoId5hP-izFk_QqvSIy4sEGTBWTq5-0ShPEBcMNw', 'ACRh3goIMfJ8V1UkEWwS6yv3iQqEsfrcN2gldfgaIDnIPGbWMyqdajXIQsRAO9Oru01zLBQjYTk0GB_1wGy06alClQ1y-vcsgg', 'ACRh3gq9n4gO1QKID9YOUqmSE0ky56T7Zs5WJKEzJC-NQ2GYyFw0PeUhEZsVwCpJNWctXZ3MQKrgPPrLfvuEWnLZ8oeiVxTz-A', 'ACRh3gr4eRK1mT6LfOKlceCUcV6Kf081gCWO4zeRzI_ON3vB7Jg2JNN5PKHKU5DkIvXOYoc9CJymHzwYy8bPOhnn5k6TrZ_d6Q', 'ACRh3gqvU6qMb1ZGlrixH45hXGLyG-afjt7OlM_-nYA1PfeYYmEs_PINYank5Hyv2W9NjZGiLs87bn_z0wdTgECUMqmPP758Sw', 'ACRh3goy6Wh56ox976n4nB3-Phe99D2nuzdgznvRZ4bRgMg1C0JlXA9mkCOM2L25KUpyWa9LlWBxmOnF6hv1aT-WhRkjlRQzfA', 'ACRh3greqz-yqX5_F9KgwXiIrGAVMGCR_BNQlmmBm2PVjhprWgamYSUzUTL46FticjkMJQO5fXNB8gbJ4YZlsKrIJYxuQgvL8g', 'ACRh3gpHVjdhpCuvnLPzYpUhalJtpwRNmldOj87lwIIaSUaqqy6uDNiFNcAdg1WP2qdj8LrysVlw_Edbk-6g_tLfPTHt1BXVIg', 'ACRh3gqAdLYp0qRNQtDf2-fG-vQNlLl4AdSq0aGdFZREgvHfGrq2I23N6K1rEtvQ1l-y_kJbVwNEXAPKGyUqLXvphu-mImmrIA']
01-17 11:53:11 gphotos.data INFO     Saving Database ...
01-17 11:53:11 gphotos.data INFO     Database Saved.
01-17 11:53:11 gphotos      ERROR    
Process failed.
Traceback (most recent call last):
  File "/home/stefan/gphotos-sync/gphotos/Main.py", line 209, in main
    self.start(args)
  File "/home/stefan/gphotos-sync/gphotos/Main.py", line 170, in start
    self.google_photos_sync.download_photo_media()
  File "/home/stefan/gphotos-sync/gphotos/GooglePhotosSync.py", line 293, in download_photo_media
    response = self._api.mediaItems.batchGet.execute(mediaItemIds=batch_ids)
  File "/home/stefan/gphotos-sync/gphotos/restclient.py", line 42, in execute
    r = Utils.retry(5, self.do_execute, body, path, query_args)
  File "/home/stefan/gphotos-sync/gphotos/Utils.py", line 63, in retry
    raise last_e
  File "/home/stefan/gphotos-sync/gphotos/Utils.py", line 54, in retry
    res = func(*arg, **k_arg)
  File "/home/stefan/gphotos-sync/gphotos/restclient.py", line 48, in do_execute
    result.raise_for_status()
  File "/home/stefan/gphotos-sync/venv/lib/python3.7/site-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: https://photoslibrary.googleapis.com/v1/mediaItems:batchGet?mediaItemIds=ACRh3gqHl1JT7bI0JBwAZS7tb8X7lyAKstKZ-qulC_p_iUTs593VZRFqVUuRLB6zPYxUAkU5AXCtTPTk-6o3G1Gi_GfIfXo1xg&mediaItemIds=ACRh3grwr4QLizR3okwzE6o3eMIflopt2WM1fic_PqGqn3SXBpQIF3yHYjHTGLNBm1m_2CL-rPUE_UnaVFh6P19x_MrPRxQXdg&mediaItemIds=ACRh3goT1pZL18jyksVWlwxwR78F0LiufYrmASx8DhMNuH3D8vdYmKtqiLsLdW14-S-TW4RUJM-guwM2Ptbddlrra_VoTuZJ5w&mediaItemIds=ACRh3gpoWhwl6nPKM-x3NGHdCozYeK2-kvhre8FSD2BvYtaMjFARLyUz99uyL1C53cPiEi_l9Oh6cqeD1BmZLyhYPJptaHlWQg&mediaItemIds=ACRh3gqXInOYSiiOc4piYTCONw5TObVmRFvlkDOSZYOgWpjdevMUT4Xc5KJe9QWAgH0zgfFvraZEbKqqJ1unVpOwaAqSRF4ALw&mediaItemIds=ACRh3gqFt0d1v7B89jAMJfsjmD3rQ1gmWdkSD2iRKaxuOxlWnP1tPO7fWMzC9L4ExIs_7dCdcJ-pJ2QrtMYkFZuZiSNDhrxXLg&mediaItemIds=ACRh3gpQFqmjoVvqho9T6JTQjvRCwR9De8f13e6sr0FYHJd2V_AjmR2UKoCpyH5Aau5dw0hse6dH3YF297Kk3EVMLtnlJXdxpA&mediaItemIds=ACRh3goOqk8j-Cd1YlFOVsq1kHvTgx2ArkHlaGqNGCKsEFWnfrzVEcDN9XqerKXSWeeTx-2fWkisgxMb4-D0ixtWK8qqKzS1Vg&mediaItemIds=ACRh3grmrooSV-dHeAigKCCipInH7mjRyeOrJNjUDoR8X1deqlFJXMRfJyrj_17ZbZBT1dfYQ1ciLjqeW3LVqcJdzFkQPygfAw&mediaItemIds=ACRh3gozoapWXPx3rkAl3NIiI_Qu9WYSCTHwnO2eRwAMnZFIsqzOmJzShM405dFhvRGxxexZifQx0KInMP2P7tE4V4wn6zQ28Q&mediaItemIds=ACRh3goTb8UG0tbhs165u_ASH-mCU7rZtT0vetyX5_kEzOwzacEAh2Fdq5xX03_cFo8MJoBNcLApG7MxYfa1FmjJRUxgRxDCfg&mediaItemIds=ACRh3gr99yaFQivyGCfLGXQnyX1UXgJxbTtHKa9BmSYcYeKjI3qOSCq1hCzoId5hP-izFk_QqvSIy4sEGTBWTq5-0ShPEBcMNw&mediaItemIds=ACRh3goIMfJ8V1UkEWwS6yv3iQqEsfrcN2gldfgaIDnIPGbWMyqdajXIQsRAO9Oru01zLBQjYTk0GB_1wGy06alClQ1y-vcsgg&mediaItemIds=ACRh3gq9n4gO1QKID9YOUqmSE0ky56T7Zs5WJKEzJC-NQ2GYyFw0PeUhEZsVwCpJNWctXZ3MQKrgPPrLfvuEWnLZ8oeiVxTz-A&mediaItemIds=ACRh3gr4eRK1mT6LfOKlceCUcV6Kf081gCWO4zeRzI_ON3vB7Jg2JNN5PKHKU5DkIvXOYoc9CJymHzwYy8bPOhnn5k6TrZ_d6Q&mediaItemIds=ACRh3gqvU6qMb1ZGlrixH45hXGLyG-afjt7OlM_-nYA1PfeYYmEs_PINYank5Hyv2W9NjZGiLs87bn_z0wdTgECUMqmPP758Sw&mediaItemIds=ACRh3goy6Wh56ox976n4nB3-Phe99D2nuzdgznvRZ4bRgMg1C0JlXA9mkCOM2L25KUpyWa9LlWBxmOnF6hv1aT-WhRkjlRQzfA&mediaItemIds=ACRh3greqz-yqX5_F9KgwXiIrGAVMGCR_BNQlmmBm2PVjhprWgamYSUzUTL46FticjkMJQO5fXNB8gbJ4YZlsKrIJYxuQgvL8g&mediaItemIds=ACRh3gpHVjdhpCuvnLPzYpUhalJtpwRNmldOj87lwIIaSUaqqy6uDNiFNcAdg1WP2qdj8LrysVlw_Edbk-6g_tLfPTHt1BXVIg&mediaItemIds=ACRh3gqAdLYp0qRNQtDf2-fG-vQNlLl4AdSq0aGdFZREgvHfGrq2I23N6K1rEtvQ1l-y_kJbVwNEXAPKGyUqLXvphu-mImmrIA
01-17 11:53:11 gphotos      INFO     Done.
01-17 11:53:11 gphotos      INFO     Elapsed time = 0:00:05.487580
</bound></bound></bound></bound></bound>

It always crashes with this exact request (same mediaItemIds).

该提问来源于开源项目:gilesknap/gphotos-sync

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享
  • 邀请回答

8条回答

  • weixin_39613561 weixin_39613561 5月前

    The faulty item is:

    https://photos.google.com/lr/photo/ACRh3greqz-yqX5_F9KgwXiIrGAVMGCR_BNQlmmBm2PVjhprWgamYSUzUTL46FticjkMJQO5fXNB8gbJ4YZlsKrIJYxuQgvL8g

    Trying to open this in the browser leads to a 500-Error. Looking into the sqlite database that item is supposed to be an animation. I can find that exact file in Google Photos, but it has a different url/mediaItemId. I think this is an item imported via the "Shared Library" feature of Google Photos if that is relevant.

    Edit: correct URL

    点赞 评论 复制链接分享
  • weixin_39702559 weixin_39702559 5月前

    I have experienced this issue. (a) for old videos, (this is mentioned in my known issues and Google are looking at it) (b) for two files out of my entire 97K library. This needs further investigation so your info is useful. Interesting to note your ID is 99 chars, nearly all of mine are 98.

    I intend to write a 'scan' which finds all bad mediaItem Ids and blacklists them as a temp workaround.

    In the meantime, you could add your mediaItem ID to 'bad_ids' line 24 of GooglePhotosSync.py.

    Let me know if this helps. Thanks

    点赞 评论 复制链接分享
  • weixin_39613561 weixin_39613561 5月前

    My IDs are actually 98 characters. Somehow an additional char ended up in the URL I posted. I corrected that above.

    Adding that ID to bad_ids has it skipped, which is fine by me. Having bad IDs identified automatically would be awesome.

    Thanks for your help and effort. Love this tool!

    点赞 评论 复制链接分享
  • weixin_39702559 weixin_39702559 5月前

    Thanks. I'll keep this issue open until I've implemented an automatic 'bad id' cache.

    点赞 评论 复制链接分享
  • weixin_39702559 weixin_39702559 5月前

    Stefan, I have checked an update into the branch called bad_id_scan. If you start with a fresh DB (--flush-index) it should find any bad IDs in your library and write them to a YAML file gphotos.bad_ids.yaml.

    It will report errors on the first scan. On subsequent scans, it should just warn you that it skipped the files listed in the YAML. The YAML also contains the path and productUrl of the bad item to help investigate the issue.

    Please, can you give it a try and let me know if it works for you. I have 228 bad IDs out of 98000 media items, nearly all are very old videos.

    I have yet to add any system tests for this feature so appreciate some extra testing.

    点赞 评论 复制链接分享
  • weixin_39613561 weixin_39613561 5月前

    Works as intended in my tests. :+1:

    Found 1 bad ID in 22913 items which is the one I found manually before.

    点赞 评论 复制链接分享
  • weixin_39702559 weixin_39702559 5月前

    Great, thanks.

    Keeping this issue open until some system tests for the feature are added.

    点赞 评论 复制链接分享
  • weixin_39702559 weixin_39702559 5月前

    I've added a system test to check this featue.

    点赞 评论 复制链接分享

相关推荐