Buffers and Streams
A buffer is a space in memory where data is temporarily stored
Streams are a method of using data stored in a buffer (i.e., passing it on continuously)
Buffers
Buffers store data in memory so that it can be programmatically programmatically handled
If you play a 100MB video, the following will happen if you only use buffers
If you use 100MB of memory, it will always take up 100MB of memory on the server (it uses a lot of memory)
Stream
Streams divide and send data into small chunks (buffers) (e.g., 100 times of 1MB each)
Streams
Streams divide and send data into small chunks (buffers): 100 times of 1MB each)
If you play a 100MB video, here's what would happen if you only used buffers
If you use 100MB of memory, it will always take up 1MB of memory on the server (saving memory)
Why use streams
Streams reduce memory usage by sending data in smaller chunks.
Buffers, on the other hand, load the entire file into memory and process it, which can be very memory intensive for large files.
Example of using a buffer
@app.route('/buffer')
def buffer_example():
buffer = io.BytesIO()
with open('example.txt', 'rb') as f:
buffer.write(f.read()) # read the file from that part and save it to the buffer (return: buffer)
# move the pointer of the buffer to the starting position
buffer.seek(0)
return send_file(buffer, as_attachment=True, download_name='example.txt') # send the buffered file to the client
if __name__ == '__main__':
app.run(debug=True)
Example using streams
def generate():
# read the file into a buffer in memory. with open('example.txt', 'rb') as f: # read the file into a buffer in memory.
while True:
# read that part of the file and save it to the buffer ( return: buffer 1024 bytes ) chunk = f.read(1024) # read that part of the file and save it to the buffer.
# chunk = buffer
if not chunk:
break
yield chunk
@app.route('/stream')
def stream_example():
return Response(generate(), mimetype='text/plain')
generate
function reads the contents of a file in small chunks and delivers them gradually to the client
(This means that it delivers the data to the client in a streaming manner)
Difference between buffering and streaming
In advance Receive data in advance → Buffering
Play data as soon as it is received → Streaming
( Both use a storage space called a buffer, and are divided into buffering and streaming according to the size of the temporarily stored data)
( Buffer space is large and stores a lot of data in advance: Buffering )
( Buffer space is small and data is written immediately : Streaming )
Buffering
Features:
Initially, a certain amount of data is received from the server and stored in the buffer before starting playback.
For example, when receiving a 100MB file, it will initially receive 10MB and store it in a buffer before playing the video. It then takes 1MB at a time to fill the buffer and finish playback.
You may notice a slight delay in the initial playback time.
Because it prefetches some data, the video will not be interrupted if the network is temporarily unreliable.
Streaming
Features:
Starts playback as soon as it receives data
For example, if you receive a 100MB file, it will take 1MB from the server, store it in a buffer, and play the video immediately. It then continues to play in 1MB increments.
Initial playback time is fast.
If the network is unstable, the video may stutter because of the small amount of data received.
The Bottom Line
Buffers are space
Streams are how
Reference