For buffers and streams

eg



Buffers and Streams

A buffer is a space in memory where data is temporarily stored


Streams are a method of using data stored in a buffer (i.e., passing it on continuously)


Buffers

Buffers store data in memory so that it can be programmatically programmatically handled

If you play a 100MB video, the following will happen if you only use buffers

  • If you use 100MB of memory, it will always take up 100MB of memory on the server (it uses a lot of memory)


Stream

Streams divide and send data into small chunks (buffers) (e.g., 100 times of 1MB each)


Streams

Streams divide and send data into small chunks (buffers): 100 times of 1MB each)

If you play a 100MB video, here's what would happen if you only used buffers

  • If you use 100MB of memory, it will always take up 1MB of memory on the server (saving memory)


Why use streams

Streams reduce memory usage by sending data in smaller chunks.

Buffers, on the other hand, load the entire file into memory and process it, which can be very memory intensive for large files.



Example of using a buffer

@app.route('/buffer')
def buffer_example():
    buffer = io.BytesIO()
    with open('example.txt', 'rb') as f:
        buffer.write(f.read()) # read the file from that part and save it to the buffer (return: buffer)
    
    # move the pointer of the buffer to the starting position
    buffer.seek(0)
    
    return send_file(buffer, as_attachment=True, download_name='example.txt') # send the buffered file to the client

if __name__ == '__main__':
    app.run(debug=True)


Example using streams

def generate():
    # read the file into a buffer in memory. with open('example.txt', 'rb') as f: # read the file into a buffer in memory.
        while True:
            # read that part of the file and save it to the buffer ( return: buffer 1024 bytes ) chunk = f.read(1024) # read that part of the file and save it to the buffer.
                                 # chunk = buffer
            if not chunk:
                break
            yield chunk

@app.route('/stream')
def stream_example():
    return Response(generate(), mimetype='text/plain')

generate function reads the contents of a file in small chunks and delivers them gradually to the client

(This means that it delivers the data to the client in a streaming manner)

Difference between buffering and streaming

In advance Receive data in advance → Buffering

Play data as soon as it is received → Streaming

( Both use a storage space called a buffer, and are divided into buffering and streaming according to the size of the temporarily stored data)

( Buffer space is large and stores a lot of data in advance: Buffering )

( Buffer space is small and data is written immediately : Streaming )


Buffering

  • Features:

    • Initially, a certain amount of data is received from the server and stored in the buffer before starting playback.

    • For example, when receiving a 100MB file, it will initially receive 10MB and store it in a buffer before playing the video. It then takes 1MB at a time to fill the buffer and finish playback.

    • You may notice a slight delay in the initial playback time.

    • Because it prefetches some data, the video will not be interrupted if the network is temporarily unreliable.

Streaming

  • Features:

    • Starts playback as soon as it receives data

    • For example, if you receive a 100MB file, it will take 1MB from the server, store it in a buffer, and play the video immediately. It then continues to play in 1MB increments.

    • Initial playback time is fast.

    • If the network is unstable, the video may stutter because of the small amount of data received.





The Bottom Line

Buffers are space

Streams are how





Reference

버퍼와 스트림의 이해

image ref (Image ref)

May 27, 2024 Views 129