How to Implement Real-Time Video Stabilization Using OpenCV
Video stabilization is a technique that reduces unwanted motion and shaking in video footage. Handheld shooting, vibration, and motion can all cause unsteady camera movements. Video stabilization produces a smoother-looking video.
The primary goal of video stabilization is to estimate the camera’s motion between consecutive frames. The process can then apply appropriate transformations to align the frames. This minimizes the perceived movement.
Setting Up Your Environment
Start bycreating a virtual environmentto ensure the packages you install to run the program do not conflict with existing ones. Then run this terminal command to install the required libraries:
This command installs NumPy and OpenCV libraries.NumPy provides tools for numerical taskswhile OpenCV deals with computer vision tasks.
The full source code is available in aGitHub repository.
Importing the Required Libraries and Defining Three Crucial Functions
Create a new Python file and give it a name of your liking. Import NumPy and OpenCV libraries at the beginning of the script.
Importing these libraries will enable you to use their functions in your code.
Next, define three functions that will be crucial for the stabilization process.
The calculate_moving_average Function
Create a function and name itcalculate_moving_average. This function will compute the moving average of a given curve using the radius you specify. It employs a convolution operation with a specified window size and a uniform kernel. This moving average helps to smooth out fluctuations in the trajectory.
The function returns a smooth curve. It helps to reduce noise and fluctuations in the curve. It does this by averaging the values within the sliding window.
The smooth_trajectory Function
Create another function and name itsmooth_trajectory. This function will apply the moving average on each dimension of the trajectory. It will achieve this by creating a smoothed copy of the original trajectory. This will further improve the stability of the video.
Thesmooth_trajectoryfunction returns a smoothened trajectory.
The fix_border Function
Create a final function and name itfix_border. This function will fix the border of the frame by applying a rotation and scaling transformation. It takes the input frame, calculates its shape, constructs a transformation matrix, and applies the transformation to the frame. Finally, it returns the fixed frame.
Thefix_borderfunction ensures that the stabilized frames do not have any border artifacts caused by the stabilization process.
Initializing Video Stabilization and Taking the Input
Start by setting the radius that the trajectory smoothing function will use.
Then, pass in the video path of the shaky video you want to stabilize.
Get the shaky video’s properties:
Set the output format. This is the format in which the program will save the stabilized video with. You can use anycommon video formatyou like.
Finally, initialize the video writer:
The extension of the file name you pass to the video writer should be the same as the one you set in the output format.
Reading and Processing Frames
The first step of processing the shaky video starts here. It involves reading frames from the input video, calculating transformations, and populating the transforms array.
Start by reading the first frame.
Then initialize the transformation array. It will store information for each frame.
Finally, you need to calculate the optical flow between consecutive frames. Then, estimate affine transformation between the points.
The loop iterates over each frame (except the last frame) to calculate transformations. It calculates optical flow between consecutive frames using the Lucas-Kanade method.cv2.goodFeaturesToTrackdetects feature points in the previous frameprev_gray. Then,cv2.calcOpticalFlowPyrLKtracks these points in the current framecurr_gray.
Only the points with a status of 1 (indicating successful tracking) help in estimating an affine transformation matrix. The code updates theprev_grayvariable with the current grayscale frame for the next iteration.
Smoothing the Trajectory
You need to smoothen the trajectory obtained from the transformations to achieve a stable result.
The above code calculates the trajectory of the camera motion and smoothens it.
Stabilizing and Writing Frames
The final step is to stabilize the frames and write the stabilized video into an output file.
Start by resetting the video capture. This ensures future operations will read from the start of the video.
Then stabilize the video by processing each frame.
The above code stabilizes each frame using the calculated transformations, including translation and rotation adjustments. It then combines the stabilized frames with the original ones to provide a comparison.
Releasing the Video Capture and Writer
Finalize the program by releasing the video capture and writer objects.
This code also closes any open windows.
Final Program Output
The output of the program will look something like this:
And here’s an example of the stabilized video:
The output shows the comparison between the shaky video and the stabilized one.
Explore OpenCV Capabilities
You can apply OpenCV in many fields that involve computer vision. This is because it offers a wide range of functionalities. You should explore its capabilities by working on more projects that involve computer vision. This will introduce you to new concepts and give you new areas to research on.
You may not be able to pass yourself off as a great artist with this program, but you’re able to still use it to produce some impressive-looking imagery.
Some subscriptions are worth the recurring cost, but not these ones.
You don’t need to fork out for expensive hardware to run an AI on your PC.
Windows is great, but adding this makes it unstoppable.
Every squeak is your PC’s way of crying for help.
The fix was buried in one tiny toggle.
