Adding ROS Raspberry Pi 4B Camera

This post describes how to get the Raspberry Pi 4B camera working under ROS. Try following these steps to get it working on your Pi 4B-based robot too!

Where to Begin?

The basic config of the robot is:

• Raspberry Pi 4B with 2 or 4 GB RAM
• Ubuntu 18.04 64-bit (see: Install ROS on Raspberry Pi 4)
• ROS Melodic
• Raspberry Pi camera

The previous employed Ubiquity approach failed on Pi 4. So I fall back to trying to install similar to:

https://github.com/fpasteau/raspicam_node,

and,

https://github.com/raspberrypi/userland

The Details

Get the userland code:

git clone https://github.com/raspberrypi/userland.git /home/ubuntu/userland

Then…

cd userland
./buildme –aarch64

The build appeared to succeed.

My quitter, lame attempt to see if it worked failed:

raspicam

Command not found.

But,

ubuntu@ubuntu-pi4:~/userland$ sudo find . -name raspicam
./host_applications/linux/apps/raspicam

So the code appears to be there but is perhaps not getting built. Note the comment on ‘https://github.com/raspberrypi/userland’…

“Whilst 64-bit userspace is not officially supported, some of the libraries will work for it.”

There might be a ray of hope here:

https://github.com/6by9/userland/tree/64bit_mmal

Trying it…

git clone https://github.com/6by9/userland.git

Step-by-step commands:

cd src/
git clone https://github.com/6by9/userland.git
cd userland/
git checkout 64bit_mmal
./buildme –aarch64
ls /opt/vc/bin/

raspivid
/opt/vc/bin/raspivid
find . -name libmmal_core.so

This now runs:

LD_PRELOAD=”/opt/vc/lib/libmmal_vc_client.so /opt/vc/lib/libvcsm.so /opt/vc/lib/libmmal_core.so /opt/vc/lib/libmmal_util.so” /opt/vc/bin/raspistill -o cam.jpg

But gives errors:

mmal: Cannot read camera info, keeping the defaults for OV5647
mmalipc: mmal_vc_dump_client_components: mmal_vc_dump_client_components: 0 entries in use
mmalipc: mmal_vc_dump_client_contexts: mmal_vc_dump_client_contexts: 0 entries in use
mmal: mmal_vc_component_create: failed to create component ‘vc.ril.camera’ (1:ENOMEM)
mmal: mmal_component_create_core: could not create component ‘vc.ril.camera’ (1)
mmal: Failed to create camera component
mmal: main: Failed to create camera component
mmal: Camera is not enabled in this build. Try running “sudo raspi-config” and ensure that “camera” has been enabled

Fixing the Runtime Errors

So next… I added:

start_x=1
gpu_mem=128

to ‘/boot/firmware/config.txt’.

And added:

SUBSYSTEM=vchiq,GROUP=video,MODE=0660

to ‘/etc/udev/rules.d/10-vchiq-permissions.rules’.

sudo vi /etc/udev/rules.d/10-vchiq-permissions.rules
sudo usermod -a -G video ubuntu
sudo reboot
LD_PRELOAD=”/opt/vc/lib/libmmal_vc_client.so /opt/vc/lib/libvcsm.so /opt/vc/lib/libmmal_core.so /opt/vc/lib/libmmal_util.so” /opt/vc/bin/raspistill -o cam.jpg
680 LD_PRELOAD=”/opt/vc/lib/libmmal_vc_client.so /opt/vc/lib/libvcsm.so /opt/vc/lib/libmmal_core.so /opt/vc/lib/libmmal_util.so” /opt/vc/bin/raspivid -o video.h264 -t 10000

(Note: there are some messages to the console that don’t inspire confidence, but they seem to actually be harmless!)

export LD_LIBRARY_PATH=/opt/vc/lib
/opt/vc/bin/raspivid -o video.h264 -t 10000

sudo ln -s /opt/vc/bin/raspivid /usr/bin/raspivid
raspivid -o video.h264 -t 10000

(added ‘export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/vc/lib’ to .bashrc)

sudo apt install ffmpeg

Direct encode to .mp4:

raspivid -ih -t 0 -o – | ffmpeg -i – -vcodec copy test.mp4

Enable ROS raspicam_node

To enable raspicam_node, do the following steps:

cd catkin_ws/src/
git clone https://github.com/fpasteau/raspicam_node.git raspicam
cd ..
catkin_make
source devel/setup.bash
export LD_PRELOAD=”/opt/vc/lib/libmmal_vc_client.so /opt/vc/lib/libvcsm.so /opt/vc/lib/libmmal_core.so /opt/vc/lib/libmmal_util.so”

rosrun raspicam raspicam_node _height:=1280 _width:=960

In another terminal:

rosservice call /camera/start_capture

Success!

With the above in place and working, getting the video presented in ROS is via standard practice.

Troubleshooting

If you run into problems, it would be best to start from a fresh install of Ubuntu/ROS, see:

Install ROS on Raspberry Pi 4B

Let me know your experience in the comments…