weixin_39583751
weixin_39583751
2020-12-27 07:22

Stereo vision

Hi

A sense of depth is quite crucial for many real world tasks. Would that make sense to include stereo vision for the agent?

该提问来源于开源项目:beyretb/AnimalAI-Olympics

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享
  • 邀请回答

6条回答

  • weixin_39987926 weixin_39987926 4月前

    Fun fact, we reverse engineered the Unity files and found that they do have a "left", "right" and "mid" camera, so this indeed was at one point an idea at the least.

    点赞 评论 复制链接分享
  • weixin_39860280 weixin_39860280 4月前

    Hello,

    This is actually something we discussed quite a bit. Funny enough (as mentionned), the stereo camera are on the agent already, but we're not actually rendering them.

    The reason for that is the same as this issue: speed. Basically, each time we add an extra rendering to the observations we slow down training significantly.

    This is typically a point where we would love to hear from participants. To what extent are you willing to give-up training speed to get better visual observation (higher resolution and/or more cameras) ?

    Keep in mind this will also impact testing speed and therefore will shorten the inference time we'll give you, as we unfortunately don't have infinite compute.

    点赞 评论 复制链接分享
  • weixin_39583751 weixin_39583751 4月前

    Thanks for the reply.

    Tbh I don't really know at this point if it worth having it. I would say it's definitely a good idea to have them as an option and people will see if it makes sense to trade off speed to resolution or stereo vision.

    In regards to the resolution, there could also be training approaches that progressively increases the resolution to get a higher fidelity input signal (similarly to the progressive GAN approach), so having it as an option could be beneficial.

    点赞 评论 复制链接分享
  • weixin_39860280 weixin_39860280 4月前

    Hello!

    We've added an option to change the resolution of the agent. It's still square and can vary between 8x8 and 512x512 (defaults to the Atari 84x84 as this is what you'll be tested on).

    Stereo might come later but I can't guaranty anything in terms of timing.

    点赞 评论 复制链接分享
  • weixin_39583751 weixin_39583751 4月前

    Great, thanks.

    Let us know when you know more about the timeline for the stereo vision.

    点赞 评论 复制链接分享
  • weixin_39860280 weixin_39860280 4月前

    Hello,

    We have now released the source code for the environment here. You can build your own training environment from it and add observations (for example extra cameras) to your agent. It does take a bit of learning to do so, but you can find some great documentation on the ML Agents repo.

    You will find the two cameras was referring to, attached to the agent.

    Please note that we do not support the environment repo at the moment.

    点赞 评论 复制链接分享

相关推荐