Camera Streaming via Ethernet from Petalinux on Zedboard to a Client PC

The main goal of this project is to stream live images from a camera connected to a Zedboard running Petalinux, via TCP sockets to a client who runs a python script in order to print the image streaming. The server who runs on Petalinux is written in C and is used to control the camera by taking snapshots sending them to the client. The hardware code is written in both Verilog and VHDL.

Camera2

Video Demonstration

Let’s see the equipment we used for the purposes of this project:

  1. Zedboard development kit
  2. OV7670 camera module
  3. Petalinux 2015.2
  4. Vivado and SDK 2015.2 (following this compatibility guide)
  5. Ethernet Cable

Camera3

This is our project Block Diagram

DiagramCamera

You can find the instructions on how to establish a connection via Ethernet between your PC and a Zedboard running Petalinux on chapter H and I of this post by embeddedcentric.

Let’s now begin explaining the hardware part of this project which code and Petalinux boot images are available here.

The Hardware

block_camera

First of all we modify the processing system according to the bring up guide for petalinux.

Now to the cameraIp module. We use some of the code of this project by Mike Field and with some additions we managed to add the functionality for the camera to take snapshots. We basically removed the vga part of the code, created the snapshot module and also we modified the top module according to our current specifications. All this modules were packaged in an Axi-4-Lite Ip to work with the processing system. The control signals and the captured image were used in the Axi bus and the camera function signals were made external so as to be connected to the pmod pins of the Zedboard.

The logic part of this Ip works like this:

  1. The control signal of the arm processor arrives
  2. The camera freezes and a snapshot is stored in a Bram of the Ip with the dimensions of 640×480 in 8-bit format.
  3. At every access of the data from the processor, the hardware converts the pixels format to 24-bit(the client uses a 24-bit format to print out the images) and propagates 4 pixels to the output registers.
  4. After a successful image reading a control signal from the processor arrives and the camera unfreezes.

Now the hardware part is completed and we used these steps to create the hardware platform to run petalinux on it.

The Petalinux Server Software

We created the server application to work this way:

  1. The server opens a TCP socket and waits for a connection from the client
  2. When connected it waits for the client request signal to start streaming
  3. When the request signal arrives, the server freezes the image through a control signal sent to the specific address of the cameraIp interface
  4. The server reads the image from the cameraIp interface and stores it in a array
  5. When finished, the image array is sent through the socket port to the client
  6. Unfreezes the camera and waits for another request to run again from step 3

The Client Software

Finally we created a simple application in python which receives the image from the TCP socket port and renders it in a graphical window. We use the pygame library of python to create the image window and update it with the new images from the socket port.

Creating Apps for Custom Hardware in Petalinux

In this post we will create an application in C that controls the hardware we created in a previous one. The application reads via polling the buttons value and propagates it to the counter interface. Then according to the buttons pressed, the counter functions as described in the previous post. The counter output is finally written at the leds interface.

LEDs

Lets now explain how the code works.

#define BUTTONS_BASE_ADDR      0x41210000
#define LEDS_BASE_ADDR              0x43C00000
#define GRAY_BASE_ADDR              0x43C10000

First of all, we must define the addresses of the peripherals interface. These addresses are set from the Vivado project when we create the peripherals.

void             *ptr_buttons, *ptr_gray, *ptr_led;
unsigned    page_addr_buttons, page_addr_gray, page_addr_led;
unsigned    page_offset_buttons, page_offset_gray,page_offset_led;
unsigned    page_size = sysconf(_SC_PAGESIZE);

After that, we declare the above variables in order to communicate with the peripherals.

page_addr_buttons      = (BUTTONS_BASE_ADDR & ~(page_size-1));
page_offset_buttons   = BUTTONS_BASE_ADDR – page_addr_buttons;

page_addr_gray            = (GRAY_BASE_ADDR & ~(page_size-1));
page_offset_gray    = GRAY_BASE_ADDR – page_addr_gray;

page_addr_led        = (LEDS_BASE_ADDR & ~(page_size-1));
page_offset_led        = LEDS_BASE_ADDR – page_addr_led;

int    fd;

fd = open(“/dev/mem”,O_RDWR);

We open a file descriptor which is essential for mapping the peripheral physical address to a virtual one. Here is how the mapping is done using the mmap function

ptr_buttons = mmap(NULL, page_size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, (BUTTONS_BASE_ADDR & ~(page_size-1)));

ptr_gray     = mmap(NULL, page_size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, (GRAY_BASE_ADDR & ~(page_size-1)));

ptr_led     = mmap(NULL, page_size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, (LEDS_BASE_ADDR & ~(page_size-1)));

Now, in order to read from a peripheral(for example the buttons) we use the following command:

counter = *((unsigned *)(ptr_gray+page_offset_gray+4));

The +4 offset in the command is because in the gray counter module in the vivado project the output of the module is stored in the slv_reg1(for example if we wanted to access the slv_reg3 we would have to use a +12 offset).

Now in order to write we simply use this command:

*((unsigned *)(ptr_led+page_offset_led)) = counter;

Conclusion:

In order to write an application which control the custom designed peripherals you have to know the physical address of them and map it to a virtual one. Then use these virtual addresses and offsets in a simple command to read and write to those peripherals.

Zedboard Petalinux with Custom Hardware

In this post we will design our custom hardware for the Zedboard that includes a simple gray counter, a led Ip and the buttons which interact with each other via an Axi-4-Lite protocol. After that we developed a software application which uses a polling method to read the buttons input and we use that to control the gray counter(for example the upper one freezes the counter and the down one resumes it). The state of the counter is propagated through the software to the leds of the Zedboard.

Untitled.png

Video Demonstration

Here you can find the Vivado project which is written in verilog and the software app in C. We will dive in depth on how to create those apps in order to use them on petalinux in a next post.

This is the block_design of the Vivado project.

block1

According to the Xilinx guide and the Svenand blog we must configure the zynq processing system according to the bring up guide in order our designed hardware to be bootable. So after a double-click on the ZYNQ7 Processing System block:

config1io1

If you want to use the Ethernet port of the Zedboard with Petalinux you have to make the changes as shown in the following pictures.

eth01eth02

At last you have to change the PS clock frequency from 100MHz to 50MHz.clock1

After all the above steps are done, now we generate the bitstream. Then export the hardware including the bitstream and launch SDK. After it finishes importing the hardware specification close the SDK and copy the .hdf file from the .sdk/ directory to the petalinux project/ directory.

Now we are ready to create a bootable image which includes the hardware we just created!! These steps are to be followed for every hardware specification you want to import to the petalinux kernel.

1)Create a Petalinux Project

petalinux-create --type project --template zynq --name <project-name>

2)Get the hardware description from the .hdf file

petalinux-config --get-hw-description -p <project-name-directory/>

project_new

After this command the kernel configuration window appears. We don’t want to change anything here so we simply select exit.

3) Enter in project directory <project-name/> and in order to add some apps that work with the hardware you designed.

petalinux-create -t apps --name <app name>

ATTENTION! You have to replace  the <app name.c> code with yours in the app directory (components/apps/).

4)In order to include your app in the Petalinux Kernel

petalinux-config -c rootfs

Then you see the following window.

Select Apps and enable your app.

app1app2

5) Build your new Kernel

petalinux-build

After this the image.ub is created in the images/linux/ directory

6) Create the bootable file

petalinux-package --boot --format BIN --force --fsbl images/linux/zynq_fsbl.elf --fpga images/linux/<project_name.bit> --uboot

Now the BOOT.BIN file is created in the project directory.

7) Copy the BOOT.bin & image.ub to an empty SD card(not in any folder).

8) Connect to the Zedboard uart interface as shown in the pictures.

File_001File_000

9) Open a serial console(gtkterm, minicom etc.) to get access to the board(port:ACM0-baud:115200). Needs superuser.

10) Insert the SD to the Zedboard and press PS-RST to see all the booting screen messages.

11) Login to the zedboard  user: root
pass: root

12) Run your app just by typing its name on the console.

For example run:

grayled

13) Have fun!!

Screenshot from 2016-07-26 16:39:18booted_gray

 

Authors:

Nikos Katsaros

Nikos Patsiatzis