2020-11-25 22:10

User generated av.packet

I have jpeg buffers and I would like to mux them into a container.

I believe all I need is write each jpeg buffer into a av.packet and mux that into a stream. As I see it right now av.packet is not designed to be modified by the user.

Simply adding a property and using the python buffer protocol should do the trick, no?


  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享
  • 邀请回答


  • weixin_39988164 weixin_39988164 5月前

    I added a property to packet.Packet:

    property payload:
         def __set__(self,uint8_t[::1] payload):
            self.struct.data = &payload[0]
            self.struct.size = len(payload)

    Creating the packet works.

    Here is how I use it:

    container = av.open(self.file_loc,'w')
    video_stream = container.add_stream('mjpeg')
    self.configured = False
    def write_video_frame(self, input_frame):
        if not self.configured:
            self.video_stream.height = input_frame.height
            self.video_stream.width = input_frame.width
            self.configured = True
        packet = Packet()
        packet.payload = input_frame.jpeg_buffer.view()

    right now start_encoding (called through mux() ) fails with 'invalid argument error'.

    Any ideas?

    点赞 评论 复制链接分享
  • weixin_39541189 weixin_39541189 5月前

    I can see I need to learn all the ins and outs of Cython's typed memory views. My understanding was that they only generally worked for writeable buffers, and so the av.utils.ByteSource class was born.

    As you can see I've made similar modifications in a branch, and I get playable output. (As long as you look after I post this, since I've rewritten history on that branch since it first appeared.)

    There are some tricky thing to consider re: memory management. Perhaps a Packet should be able to take a buffer-able input, use that buffer, and somehow pass ownership of that pointer off to the stream until we are sure we are done with it.

    点赞 评论 复制链接分享
  • weixin_39988164 weixin_39988164 5月前


    ok, so all I needed to do was give the correct pix fmt 'yuvj422p' to make it work. (Your byte source is very nice because it allows using anything that supports to python buffer protocol if I understand correctly.

    Yes memory ownership is a very tricky subject with interfacing c libs. In your example to keep a reference of the owner around forever. That wont work in a real application. Any idea for how long we need to keep the buffer alive?

    点赞 评论 复制链接分享
  • weixin_39541189 weixin_39541189 5月前

    The Plane.update method copies the input so we can immediately forget it. I figure the Packet.update could do the same.

    I would, however, like to see the plane and packet simply copy the pointer, but we would need to have assurances about the lifetime of a packet or a plane. The docs say "the packet data is typically backed by some static buffer somewhere and is only valid for a limited time", so I find it likely with some digging we could figure it out.

    点赞 评论 复制链接分享
  • weixin_39988164 weixin_39988164 5月前

    Using pointers would be elegant but I m not sure how this will work. Here an exempt form the docs on av_interleaved_write_frame:

    The packet containing the data to be written. pkt->buf must be set to a valid AVBufferRef describing the packet data. Libavformat takes ownership of this reference and will unref it when it sees fit. The caller must not access the data through this reference after this function returns. (...)

    It seems to me we may never know when the data can be freed....

    点赞 评论 复制链接分享
  • weixin_39541189 weixin_39541189 5月前

    We would be able to test if it were safe to free, as the reference count would drop. If we hold an explicit reference, we can periodically check it. The "problem" I see is that there is no callback to check. However, actually usage patterns may reveal that to not be a problem at all.

    Lets say we create an av.Buffer which wraps an AVBufferRef. The various PyAV objects which accept it would need to take responsibility to transfer those two together, such that if the buffer is given to av_interleaved_write_frame it puts the Python wrapper in a private list somewhere on the container that is polled when there is activity on the container, and unref's it when it holds the last reference.

    In the meantime, how about I setup av.Packet to have the same buffer interface as av.Plane?

    点赞 评论 复制链接分享
  • weixin_39541189 weixin_39541189 5月前
    点赞 评论 复制链接分享
  • weixin_39953236 weixin_39953236 5月前

    I think this issue can be closed, I believe it's fixed. I'm successfully using Packet as simply as:

    packet = Packet(some_bytes)
    点赞 评论 复制链接分享
  • weixin_39953236 weixin_39953236 5月前

    As indicate above, I believe that this issue is fixed, you can construct Packet instances in version 0.5.3.

    Feel free to re-open if you believe this is not the case.

    点赞 评论 复制链接分享