Jetson 16 Controlling Photo Capture with Buttons

This tutorial teaches how to control the camera for taking photos and recording videos by adding buttons to the page. Similar to previous tutorials, images are by default saved in the /ugv_jetson/templates/pictures/ folder, and videos are saved in the /ugv_jetson/templates/videos folder.

Preparation

Since the product automatically runs the main program at startup, which occupies the camera resource, this tutorial cannot be used in such situations. You need to terminate the main program or disable its automatic startup before restarting the robot.
It's worth noting that because the robot's main demo uses multi-threading and is configured to run automatically at startup through crontab, the usual method sudo killall python typically doesn't work. Therefore, we'll introduce the method of disabling the automatic startup of the main program here.
If you have already disabled the automatic startup of the robot's main demo, you do not need to proceed with the section on Terminate the Main Demo.

Terminate the Main Demo

1. Click the "+" icon next to the tab for this page to open a new tab called "Launcher."

2. Click on "Terminal" under "Other" to open a terminal window.

3. Type bash into the terminal window and press Enter.

4. Now you can use the Bash Shell to control the robot.

5. Enter the command: sudo killall -9 python.

Example

The following code block can be run directly:

1. Select the code block below.

2. Press Shift + Enter to run the code block.

3. Watch the real-time video window.

4. Press STOP to close the real-time video and release the camera resources.

If you cannot see the real-time camera feed when running:

Click on Kernel -> Shut down all kernels above.
Close the current section tab and open it again.
Click STOP to release the camera resources, then run the code block again.
Reboot the device.

Notes

If you are using a USB camera, you need to uncomment the line frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB).

Running

When the code block is executed, you can take a photo by clicking on PHOTO.

import cv2 # Import the OpenCV library for image processing
from picamera2 import Picamera2 # For accessing Raspberry Pi Camera library
import numpy as np # For mathematical calculations
from IPython.display import display, Image #  Import to display images in Jupyter Notebook
import ipywidgets as widgets # Import to create interactive interface widgets like buttons
import threading # Import to create new threads for asynchronous task execution

import os, time # Import for file and directory operations and time-related functions

time_intervel = 3 # Set the time interval for taking photos (seconds)

photo_path = '/home/ws/ugv_jetson/static/' # Set the directory path to store photos and videos

# Create "Stop" button for users to click on it to stop capturing video and photos  
# ================
stopButton = widgets.ToggleButton(
    value=False,
    description='Stop',
    disabled=False,
    button_style='danger', # Set button style 'success', 'info', 'warning', 'danger' or ''
    tooltip='Description',
    icon='square' # Set button icon (FontAwesome names without the `fa-` prefix)
)

# Create a "Stop" button for users to stop video capture and photo taking
# ================
photoButton = widgets.ToggleButton(
    value=False,
    description='Photo',
    disabled=False,
    button_style='danger', # 'success', 'info', 'warning', 'danger' or ''
    tooltip='Description',
    icon='square' # (FontAwesome names without the `fa-` prefix)
)

# Set the time interval for continuous shooting (seconds)
time_interval = 3
photo_num_count = 0 #Initialize the photo counter
capture_lock = threading.Lock()
last_photo_time = time.time() #Record the last time a photo was taken

def photo_button_clicked(change, frame):
    global photo_num_count
    if change['new']: # When the "Photo" button is clicked
        photo_num_count = photo_num_count + 1 # Photo counter plus 1
        photo_filename = f'{photo_path}photo_{photo_num_count}.jpg'  # Set the path and filename for 
        cv2.imwrite(photo_filename, frame) # Save the photo
        print(f'{photo_num_count} photos saved. new photo: {photo_filename}') # Print photo save information
        photoButton.value = False # Reset the status of the "Photo" button

# Define a display function to capture and display video frames and respond to photo capture requests
# ================
def view(stop_button, photo_button):
    # If you are using a CSI camera you need to comment out the picam2 code and the camera code
    # Since the latest versions of OpenCV no longer support CSI cameras (4.9.0.80), you need to use picamera2 to get the camera footage
    
    # picam2 = Picamera2() # Create Picamera2 example
    # picam2.configure(picam2.create_video_configuration(main={"format": 'XRGB8888', "size": (640, 480)})) # Configure camera parameters
    # picam2.start()

    camera = cv2.VideoCapture(-1) # Create camera example
    #Set resolution
    camera.set(cv2.CAP_PROP_FRAME_WIDTH, 640)
    camera.set(cv2.CAP_PROP_FRAME_HEIGHT, 480)
    
    display_handle=display(None, display_id=True) # Create a display handle to update the displayed image
    i = 0
    while True:
        # frame = picam2.capture_array() # Capture a frame from the camera
        _, frame = camera.read() # Capture a frame from the camera

        photoButton.observe(lambda change: photo_button_clicked(change, frame), names='value') # Listen for clicks on the "Photo" button
            
        _, frame = cv2.imencode('.jpeg', frame) # Encode the frame as JPEG format
        display_handle.update(Image(data=frame.tobytes()))
        if stopButton.value==True:
            # picam2.close() # If yes, close the camera
            cv2.release() # if yes, close the camera
            display_handle.update(None)

            
# Display the "Stop" and "Photo" buttons and start a new thread to execute the display function
# ================
display(stopButton)
display(photoButton)
thread = threading.Thread(target=view, args=(stopButton, photoButton,))
thread.start()

Here's something to note: due to stability issues with JupyterLab components used in this example, pressing the "Photo" button may result in saving multiple photos. You can navigate to /ugv_jetson/templates/pictures/ in the left sidebar of JupyterLab to view the captured photos.

Navigation menu