12 Image Transmission Based on Flask

From Waveshare Wiki
Jump to: navigation, search

This chapter introduces how to use Flask to create a web application for displaying real-time video from the robot's camera. Due to the cross-platform nature of web applications, users can watch the camera's real-time video on devices such as smartphones, PCs, tablets, etc., through a browser, achieving wireless image transmission functionality.

What is Flask?

Flask is a lightweight web application framework used to quickly build web applications using Python.

  • Lightweight: Flask is a lightweight framework with a relatively small core library, but it offers enough flexibility and extensibility for developers to choose and add the extensions and libraries they need.
  • Simple and Easy to Use: Flask is designed to be simple and easy to use. Its API is clear and well-documented, allowing developers to quickly get started and build web applications rapidly.
  • Routing System: Flask uses decorators to define URL routes, mapping requests to corresponding handler functions. This makes creating different pages and handling different requests intuitive and straightforward.
  • Template Engine: Flask integrates with the Jinja2 template engine, making it easier to build dynamic content within the application. The template engine allows you to embed dynamically generated content in HTML.
  • Integrated Development Server: Flask comes with a simple integrated development server for easy development and debugging. However, in production environments, it is recommended to use more powerful web servers such as Gunicorn or uWSGI.
  • Plugins and Extensions: Flask supports many plugins and extensions for adding additional functionality, such as database integration, authentication, form handling, etc.
  • RESTful Support: Flask provides good support for RESTful-style APIs, making it simple to build and design RESTful APIs.
  • WSGI Compatibility: Flask is based on the Web Server Gateway Interface (WSGI), allowing it to run on many web servers that comply with the WSGI standard.
  • Active Community: Flask has a large and active community, meaning you can easily find extensive documentation, tutorials, third-party extensions, and support.d support.

Preparation

Since the product automatically runs the main program at startup, which occupies the camera resource, this tutorial cannot be used in such situations. You need to terminate the main program or disable its automatic startup before restarting the robot.
It's worth noting that because the robot's main program uses multi-threading and is configured to run automatically at startup through crontab, the usual method sudo killall python typically doesn't work. Therefore, we'll introduce the method of disabling the automatic startup of the main program here.
If you have already disabled the automatic startup of the robot's main demo, you do not need to proceed with the section on Terminate the Main Demo.

Terminate the Main Demo

1. Click the "+" icon next to the tab for this page to open a new tab called "Launcher."
2. Click on "Terminal" under "Other" to open a terminal window.
3. Type bash into the terminal window and press Enter.
4. Now you can use the Bash Shell to control the robot.
5. Enter the command: crontab -e.
6. If prompted to choose an editor, enter 1 and press Enter to select nano.
7. After opening the crontab configuration file, you'll see the following two lines:
@reboot ~/ugv_pt_rpi/ugv-env/bin/python ~/ugv_pt_rpi/app.py >> ~/ugv.log 2>&1
@reboot /bin/bash ~/ugv_pt_rpi/start_jupyter.sh >> ~/jupyter_log.log 2>&1
8. Add a # character at the beginning of the line with ……app.py >> …… to comment out this line.
#@reboot ~/ugv_pt_rpi/ugv-env/bin/python ~/ugv_pt_rpi/app.py >> ~/ugv.log 2>&1
@reboot /bin/bash ~/ugv_pt_rpi/start_jupyter.sh >> ~/jupyter_log.log 2>&1
9. Press Ctrl + X in the terminal window to exit. It will ask you Save modified buffer? Enter Y and press Enter to save the changes.
10. Reboot the device. Note that this process will temporarily close the current Jupyter Lab session. If you didn't comment out ……start_jupyter.sh >>…… in the previous step, you can still use Jupyter Lab normally after the robot reboots (JupyterLab and the robot's main program app.py run independently). You may need to refresh the page.
11. One thing to note is that since the lower machine continues to communicate with the upper machine through the serial port, the upper machine may not start up properly during the restart process due to the continuous change of serial port levels. Taking the case where the upper machine is a Raspberry Pi, after the Raspberry Pi is shut down and the green LED is constantly on without the green LED blinking, you can turn off the power switch of the robot, then turn it on again, and the robot will restart normally.
12. Enter the reboot command: sudo reboot.
13. After waiting for the device to restart (during the restart process, the green LED of the Raspberry Pi will blink, and when the frequency of the green LED blinking decreases or goes out, it means that the startup is successful), refresh the page and continue with the remaining part of this tutorial.

Web Application Example

Note: The following code block cannot be run in Jupyter Lab

Due to potential conflicts in port usage between the Flask application and Jupyter Lab, the following code cannot be run in Jupyter Lab. The code is stored in the 12 folder within the "tutorial_cn" and "tutorial_en" directories. Inside the "12" folder, there is also a folder named "template" for storing web resources. Below are the steps to run the example:

1. Open a terminal using the method described earlier. Make sure the terminal's default path matches the file path on the left. Navigate to the 12 folder by entering "cd 12" in the terminal.
2. Start the Flask web application server using the following command: python flask_camera.py.
3. Open a web browser on a device within the same local network (or open a new tab in the browser on the same device) and enter the Raspberry Pi's IP address followed by: 5000. For example, if the Raspberry Pi's IP address is "192.168.10.104", enter "192.168.10.104:5000" in the browser's address bar. Note that it should be an English colon.
4. Use Ctrl + C in the terminal to end the running application.

Flask Example

from flask import Flask, render_template, Response  # Import Flask class, render_template function for rendering HTML templates, Response class for generating response objects
from picamera2 import Picamera2  # Import Picamera2 class from picamera2 library for accessing and controlling the camera
import time  # Import the time module for handling time-related tasks
import cv2  # Import the OpenCV library for image processing

app = Flask(__name__)  # Create a Flask application instance

def gen_frames():  # Define a generator function for generating frames captured by the camera
    picam2 = Picamera2()  # Create an instance of Picamera2

    # Configure camera parameters, set video format and size
    picam2.configure(picam2.create_video_configuration(main={"format": 'XRGB8888', "size": (640, 480)}))

    picam2.start()  # Start the camera
    while True:
        frame = picam2.capture_array()  # Capture a frame from the camera
        
        ret, buffer = cv2.imencode('.jpg', frame)  # Encode the captured frame as JPEG format

        frame = buffer.tobytes()  # Convert the JPEG image to a byte stream

        # Use yield to return the image byte stream, allowing for continuous transmission of video frames to form a video stream
        yield (b'--frame\r\n'
               b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n')

@app.route('/')  # Define the root route
def index():
    return render_template('index.html')  # Return the index.html page

@app.route('/video_feed')  # Define the video stream route
def video_feed():
    # Return a response object with video stream content, with content type multipart/x-mixed-replace
    return Response(gen_frames(), mimetype='multipart/x-mixed-replace; boundary=frame')

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000, debug=True)  # Start the Flask application, listening on port 5000 on all network interfaces, with debug mode enabled

Explanation of Key Parts of the Code

gen_frames(): This is a generator function that continuously captures frames from the camera, encodes them into JPEG format, and yields the frame bytes as part of a multipart response. The generated frames are streamed in real-time to the client.

@app.route('/'): This decorator associates the index() function with the root URL (/). When a user accesses the root URL, it renders the HTML template named '12_index.html'.

@app.route('/video_feed'): This decorator associates the video_feed() function with the '/video_feed' URL. This route is used for real-time video streaming, where frames are sent as multipart responses.

app.run(host='0.0.0.0', port=5000, debug=True): This line starts the Flask development server, listening on all available network interfaces (0.0.0.0) on port 5000. The debug=True option enables the server's debug mode.g mode.

Web

Comment:
<!doctype html>: HTML document type declaration.
<html lang="en">: The root element of the HTML document, specifying the page language as English.
<head>: Contains metadata of the document, such as character set and page title.
<!-- Required meta tags -->: HTML comment, indicating these are necessary meta tags.
<meta charset="utf-8">: Specifies that the document uses the UTF-8 character set.
<title>Live Video Based on Flask</title>: Sets the page title.
<body>: Contains the visible content of the document.
<!-- The image tag below is dynamically updated with the video feed from Flask -->: HTML comment, explaining that the image tag below will be dynamically updated to display the live video feed from Flask.
<img src="{{ url_for('video_feed') }}">: Image tag that uses the video_feed route defined in Flask to fetch the live video stream.
Retrieved from "https://www.waveshare.com/w/index.php?title=12_Image_Transmission_Based_on_Flask&oldid=81631"