SSDV Slow Scan Digital Video

For years now, high altitude balloons have been able to send live images throughout their flights and provide a more immediate experience from their tracking. This was made possible with Slow Scan Digital Video (SSDV) brought to life by Philip Heron (fsphil). It is a way of packetizing a JPEG encoded image and sending it among ordinary telemetry strings. In the original form using RTTY generally at 300-600 bauds, or nowadays at much higher speeds using LoRa. Dave Akerman has been developing the LoRa implementation and sent down hi-resolution images on a number of his flights. On the receiving side, dl-fldigi is equipped with an RTTY demodulator/decoder and automatically recognizes the packets and uploads them to an SSDV dedicated server. In this blog post I am going to describe my implementation of the RTTY version of SSDV on TT7F.

Offset	Name	Size	Description
0	Sync Byte	1	0x55
1	Packet Type	1	0x66 Normal mode, 0x67 No-FEC mode.
2	Callsign	4	Base-40 encoded. Up to 6 characters.
6	Image ID	1	Incremented by 1 for each new image.
7	Packet ID	2	The packet number within the image.
9	Width	1	Width of the image in MCU blocks (pixels / 16).
10	Height	1	Height of the image in MCU blocks (pixels / 16).
11	Flags	1	[7:6] reserved, [5:3] JPEG quality, [2] EOI flag, [1:0] subsampling
12	MCU Offset	1	Offset (bytes) to the beginning of the first MCU block in payload.
13	MCU Index	2	The number of the MCU pointed to by the offset above.
15	Payload	205/237	Payload data.
220/252	Checksum	4	32-bit CRC.
224	FEC	32	Reed-Solomon forward error correction data.

This is the basic packet structure. Each packet is 256 bytes with 205 or 237 bytes of the actual image depending on whether Reed-Solomon forward error correction is used or not. If used, it should be able to correct for up to 16 byte errors. At 300 bauds and generally low power transmissions it is a handy feature. The other contributor to successful reception is collection of packets from multiple sources at the server's side. If some packets escape reception at one station but decode and upload at another, the final image is combined from all of them.

Above is one complete SSDV packet as an example. The payload data contain only the JPEG image data without a header or any markers. The encoder takes the original jpeg image, decodes it and re-encodes it using a specific set of quantization and huffman tables before it splits it into individual packets. Since the decoder now knows which tables to use, they don't have to be transmitted with the image data saving some space. Basic information about the image, such as resolution, is included in the SSDV packet's header.

Quality	Q Factor	img1 50q	img1 90q	img2 50q	img2 90q
original		11865	33406	14693	38838
0	12.76	6400	6656	4352	4608
1	18.00	8704	8704	9216	8448
2	28.83	13312	11520	15872	12544
3	42.71	14080	14080	17152	16128
4	49.61	14336	15616	17664	17664
5	70.82	17152	20992	22784	24576
6	85.95	20992	37888	28672	43264
7	100.00	28928	63488	41216	78080

The input image to the SSDV encoder has to have width and height divisible by 16, it has to be in YCbCr color format and single scan baseline DCT mode (as opposed to multi scan progressive DCT which allows decoding the jpeg image in steps of increasing quality). Although the encoder code suggests it supports images with restart markers (FFDD and FFD0-D7), I noticed some of the decoded images had ended up garbled. The table above shows the size in bytes of all of the resulting SSDV packets created from an original jpeg image (second row) with different SSDV quality settings (values 0-7 representing adjustments to the quantization tables used to re-encode the image while packetizing it). The corresponding jpeg quality factors are in the second column. For the examples I used two different sceneries (img1, img2) taken with 50 and 90 jpeg quality factor.

Quality	Q Factor	img1 50q	img1 90q	img2 50q	img2 90q
0	12.76	25	26	17	18
1	18.00	34	34	36	33
2	28.83	51	45	62	49
3	42.71	54	55	67	63
4	49.61	55	61	69	69
5	70.82	67	81	90	96
6	85.95	83	147	114	169
7	100.00	114	251	167	313

This table continues the example and shows the number of SSDV packets per image. The figure gets slightly higher than a simple division of the decoded image size minus the header by 205 bytes of data per packet, because the packets contain only completed MCUs (the minimum coded unit of a jpeg image).

Quality	Q Factor	img1 50q	img1 90q	img2 50q	img2 90q
0	12.76	3.91	4.07	2.66	2.82
1	18.00	5.32	5.32	5.63	5.16
2	28.83	7.98	7.04	9.70	7.67
3	42.71	8.45	8.60	10.48	9.86
4	49.61	8.60	9.54	10.79	10.79
5	70.82	10.48	12.67	14.08	15.02
6	85.95	12.98	23.00	17.83	26.44
7	100.00	17.83	39.27	26.13	48.97

Transmitting one packet (256 bytes) at 300 baud takes 9.39s. Each data byte requires 11 bits (1 start bit, 8 data bits, 2 stop bits). The table shows the time in minutes to transmit the whole image. At 600 baud it would take half the time (4.69s per packet) to complete the transmission.

Quality	Q Factor	img1 50q	img1 90q	img2 50q	img2 90q
original	90.44	11865	33406	14693	38838
0	12.76	5704	5771	4048	4275
1	18.00	7498	7547	7844	7342
2	28.83	11034	9725	13133	10619
3	42.71	11635	11766	14301	13427
4	49.61	11851	12988	14646	14712
5	70.82	14244	17136	18920	20265
6	85.95	17486	30704	23953	35174
7	100.00	23967	52008	34825	64762

The last table contains the sizes (byte) of the decoded images. In case a packet doesn't make it through the missing MCUs in the final image are filled with colour according to the last DC coefficient.

This is the original image img1 90q with JPEG quality factor of 90.

 

 

The decoded images are above in order of increasing quality. A quality factor of 50 (SSDV quality of 4) seems to be a good compromise between visual impression and image size.
ARM_SSDV.h
ARM_SSDV.c
ARM_RS8.h
ARM_RS8.c
These are the libraries used on TT7F. They are the original Philip Heron's libraries with only minor changes to get them to compile in SAM3S environment. The files include only the encode functions since decoding isn't necessary in the tracker. The RS8 library contains a function to generate the Reed-Solomon error correcting symbols to append to the end of the packet.


Aptina MT9D111 camera module
On the hardware side of things I opted for a cheap 2-megapixel camera module for about €6.50 from Ebay. The MT9D111 seemed a good compromise in image quality, price and available information. I reckoned more than 2-megapixels would be pointless at the speeds I expected to transmit. I also considered a similar camera module - OV2640, but the amount of accessible information and software examples led me to choose MT9D111. The datasheet can be found at one of the sellers' website.

This is the schematic of the parts and connections the camera module needs around the receptacle (AVX 145602‐030000‐829 found on Aliexpress). It requires two power supplies (1.8V digital, 2.8V analog, PLL and I/O), a decoupling capacitor and a few pull-up resistors. I used the 2.54mm adapter board that is sold with the camera as an example of a working setup and basically mirrored it on TT7F.

The camera is then wired to ATSAM3S8B microcontroller that takes care of controlling it and sampling the image. The MCU is conveniently equipped with Parallel Capture Mode - a build-in interface for CMOS digital image sensors and high-speed parallel ADCs. Although the Parallel Capture Mode isn't essential, it significantly simplifies and speeds up the image acquisition. The camera outputs the JPEG image data on its D0 to D7 pins one byte per one pixel clock cycle. The MCU provides a clock signal to the camera on one of its PWM pins (CLK). On the VSYNC, HREF and PCLK lines the camera outputs signals informing about available data on the D0-D7 pins. VSYNC outlines an active frame, HREF an active row and PCLK is the output clock. The communication between the MCU and the camera is established via a two-wire interface (TWI) on two lines (TWD0 and TWCK0) that are pulled up to the 2.8V supply.

This is the footprint of the camera connector and some of the external parts on the bottom side of the PCB (blue). Note that in reality the connector is mirrored. This is a view from the top side of the PCB. In the image on the right is a footprint of ATSAM3S8B at the top side of the PCB (red).

This then is a composite of the two showing how the two parts are wired on TT7F. If I were designing it again, I would leave more space between the connector pads and the vias underneath the connector. The problem is that the connector's pins continue beyond the pads in the inward direction (something I hadn't noticed when I was making the footprints and pcb) and may touch the vias shorting/corrupting the signal.

This is the receptacle for the camera module. One can either desolder one from the 2.54mm adapter, or have to find a seller. I managed to come across a whole reel of them on Aliexpress about a year ago, but I can't see them on offer now. The designation is AVX 145602‐030000‐829.

This is the camera module that plugs in the receptacle. It weighs 1.44g. It was also advertised as 'with auto focus lens', however enabling Auto Focus driver doesn't seem to have any effect on the image sharpness nor can I detect any movement in the lens when the camera is taking a picture. The datasheet goes in length about auto focus, but it all assumes an external lens connected to one of the GPIOs. I am not quite sure what to think about it.

The previously mentioned problem with the pins touching vias may be clearer from this series of photos. I solved it by sticking a piece of Kapton tape in between the pins and soldering the connector on top of it.

 

The footprint for the connector probably doesn't have to have the two rows of pads so far apart. The connector in reality barely reaches across. The two holes in the corners of the footprint are also useless. So much for drawing it according to the datasheet. The two voltage regulators are ME6211 - 1.8V and 2.8V versions. The same as on the 2.54mm adapter.

And a couple of final photos with the camera plugged in. The ribbon cable allows some bending, but it would be reasonable to add additional fixation for some larger angles so the plug doesn't pop out.


Software
ARM_MT9D111.h
ARM_MT9D111.c
ARM_PIODC.h
ARM_PIODC.c
ARM_PWM.h
ARM_PWM.c
ARM_TWI.h
ARM_TWI.c
The first thing necessary to do is to provide a clock signal to the camera. The datasheet states the range of input frequencies to be 6-64MHz. In TT7F application I was limited by this 6MHz minimum and the maximum at which the Parallel Capture Mode managed to sample the image correctly. Experimentally I arrived at 10.7MHz CLK to be the fastest to be able to do so (64MHz MCK / 6 for the PWM settings). This is the frequency I use in the function PWM_init() which initializes the pin and generates the appropriate signal. By default the output frequency (PCLK - frequency at which the image data is clocked out) equals to the input frequency, however MT9D111 allows applying a divider and scale the clock down, or enable PLL and generate clock speeds of up to 80MHz. In general faster clock means faster image processing, faster sampling, less on time and thus less consumption.
The second thing to do is establishing communication with the camera. That amounts to initializing the two wire interface with TWI_init_master() at somewhere below the camera's maximum speed of CLK/16 - 100kHz should be a good universal frequency to start with. To command the camera, change settings or take a picture, MT9D111 uses register and variable writes - MT9D111_register_write(), MT9D111_variable_write() - and also reads - MT9D111_register_read(), MT9D111_variable_read(). To avoid corrupting frames in the middle of processing, the camera has shadow registers (in the form of variables) that require a refresh command or switching a context to take effect. The variables have a little catch in that one has to distinguish between writing/reading an 8-bit or a 16-bit variable (which variable is which can be found in the datasheet). Generally if a setting has both a register and a variable, set the variable. If not, set the register.
Moving onto the MT9D111 library. There is one thing I haven't mentioned yet which I do at the very beginning of the whole code, and that is running MT9D111_init(). This function switches all the VSYNC, HREF, PCLK and D0-D7 pins to appropriate peripheral function and disables the pull-up resistors that are by default on. I do this early in the script, because the SAM3S runs at 3.3V while MT9D111's tolerance on the pins is stated to be 3.1V. In reality SAM3S's pins with enabled pull-ups measure at about 3.05V so should be ok regardless. The camera should as the first command undergo a reset routine - MT9D111_reset(). After that the camera transitions to Preview mode/Context A and continually outputs a default lower resolution uncompressed image. To output a JPEG compressed image, the camera has to transition to Capture mode/Context B with a couple of settings enabled. I do this in the following function:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48

void MT9D111_mode_1(void) { MT9D111_register_write(WRITE_PAGE, 0); MT9D111_register_write(0x20, READ_MODE_B); // Binning, mirroring, column & row skipping, etc. (0x0000 - default) MT9D111_register_write(0x05, 0x015C); // Horizontal Blanking in Context B (0x015C - default) MT9D111_register_write(0x06, 0x0020); // Vertical Blanking in Context B (0x0020 - default) MT9D111_register_write(WRITE_PAGE, 1); MT9D111_register_write(0x0B, 0b11011111); // Enable JPEG clock MT9D111_variable_write(1, 2, 1, jpeg_drivers); // Enable drivers MT9D111_variable_write(7, 7, 0, res_x); // Output width for Context B MT9D111_variable_write(7, 9, 0, res_y); // Output height for Context B MT9D111_variable_write(7, 11, 0, 0); // Mode config - enable JPEG MT9D111_variable_write(7, 27, 0, 28); // First sensor-readout row (context B) PAN MT9D111_variable_write(7, 29, 0, 60); // First sensor-readout column (context B) PAN MT9D111_variable_write(7, 31, 0, 1200); // No. of sensor-readout rows (context B) PAN MT9D111_variable_write(7, 33, 0, 1600); // No. of sensor-readout columns (context B) PAN MT9D111_variable_write(7, 37, 0, 0b0000000000010001); // 0x0A(0) Row Speed Context B (delay Pixel Clock, Pixel Clock speed) MT9D111_variable_write(7, 53, 0, 0); // Lower-x decimator zoom window (context B) CROP MT9D111_variable_write(7, 55, 0, 1600); // Upper-x decimator zoom window (context B) CROP MT9D111_variable_write(7, 57, 0, 0); // Lower-y decimator zoom window (context B) CROP MT9D111_variable_write(7, 59, 0, 1200); // Upper-y decimator zoom window (context B) CROP MT9D111_variable_write(7, 61, 0, 0b0000000000000000); // Decimator control register (context B): Bit 4 - enable 4:2:0 mode MT9D111_variable_write(7, 68, 1, (jpeg_contrast << 4) | jpeg_gamma); // Gamma and contrast settings (0x42 - default) MT9D111_variable_write(7, 114, 0, OUTPUT_CONFIGURATION); // 0x0D(2) FIFO Buffer configuration 0 Context B (DS p.54) MT9D111_variable_write(7, 116, 0, 0b0000001000000011); // 0x0E(2) FIFO Buffer configuration 1 Context B (DS p.55) MT9D111_variable_write(7, 118, 1, 0b00000001); // 0x0F(2) FIFO Buffer configuration 2 Context B (DS p.55) MT9D111_variable_write(1, 33, 1, NUM_IMAGES); // Number of frames in still capture mode MT9D111_variable_write(1, 32, 1, 0b00000000); // Still image, all video options off MT9D111_variable_write(9, 6, 1, JPEG_COLOR_FORMAT); // JPEG image format MT9D111_variable_write(9, 7, 1, JPEG_CONFIGURATION); // JPEG configuration and handshaking MT9D111_variable_write(9, 8, 0, JPEG_RESTART_INTERVAL); // Restart marker interval MT9D111_variable_write(9, 10, 1, (0b10000000 | qscale_1)); // QSCALE1 MT9D111_variable_write(9, 11, 1, (0b10000000 | qscale_2)); // QSCALE2 MT9D111_variable_write(9, 12, 1, (0b10000000 | qscale_3)); // QSCALE3 MT9D111_variable_write(1, 3, 1, 2); // DO CAPTURE command MT9D111_wait_for_state(7); // CAPTURE state }

The registers inside MT9D111 are organized into Pages (0/1/2) which has to be selected before writing the desired register or variable. That is done by writing the number of the specific page into register 0xF0 - WRITE_PAGE (rows 3 and 8 in the example above). Variables come under Page 1. After that any register or variable write/read is aimed at the selected page. I start with a few registers on Page 0, but leave them default. I included it only because the 0x20 register contains bits to mirror the image and a couple more that may be used to skip columns and rows. Moving on to the Page 1, I enable JPEG clock and any drivers such as Auto Exposure, Auto White Balance, Histogram or Auto Focus I may want to use. MT9D111 allows image resolutions of up to 1600x1200, but for SSDV practicality I usually set the dimensions between 320x240 and 512x384 making sure they are divisible by 16. Cropping the image means reading data from only a limited portion of the sensor and Panning moves this window within the limits of the sensor's 1600 by 1200 pixels. Enabling the JPEG compression in ID-7 Offset-11 variable has to be done as well. ID-7 Offset-37 then allows down scaling the pixel clock. This can be achieved by enabling Variable Pixel Clock Rate in ID-7 Offset-114 as well. As a result the camera initially outputs the JPEG data at the slowest clock setting. If its JPEG buffer fills above a threshold it speeds up the clock. If it falls back below it slows the clock down again. In this manner the pixel clock alternates between three speeds based on output buffer fullness. A problem may arise when the clock speeds are set too low so the buffer overflows. The image compression aborts and has to be restarted with more adequate settings. Given that I can sample the image quite quickly I don't use the variable clock on TT7F, but it provides an option for different implementations. In ID-1 Offset-32 and 33 I select still images and tell the camera to take one image. JPEG_COLOR_FORMAT allows choosing between 4:2:2, 4:2:0 and monochrome images while JPEG_CONFIGURATION enables generating scaled quantization tables which allows selectable quality factor of the output image.

Quality	Qscale	Quality	Qscale
100	0	55	29
95	3	50	32
90	6	45	36
85	10	40	40
80	13	35	46
75	16	30	53
70	19	25	64
65	22	20	80
60	26	15	107

The camera uses Qscale variable to set the desired image quality. Its relationship to JPEG quality factor is outlined in the table above. For Quality < 50: Qscale = 1600 / Quality. For Quality >= 50: Qscale = 16 * (100 - Quality) / 25. Continuing with the JPEG example, I issue DO CAPTURE command in ID-1 Offset-3 variable and wait for the camera to transition to Capture mode/Context B with MT9D111_wait_for_state(). At this point the camera takes the programmed number of images and automatically returns back to Preview mode/Context A.

uint32_t len = CreateJpegHeader(JPEGbuffer, res_x, res_y, JPEG_COLOR_FORMAT, JPEG_RESTART_INTERVAL, qscale_1);

MT9D111 outputs only raw image data without a header and EOI (FFD9) JPEG marker. This Developer Guide, however, provides functions to construct the required header based on a few inputs that were used to setup the camera prior to taking the image. The function above puts the resulting header in JPEGbuffer and returns its length (expect 619-625 bytes).

MT9D111_mode_1();

while(PIOA->PIO_PDSR & (1 << 15));                            // wait for the end of the current frame (VSYNC HIGH->LOW)
PDC_transfer(JPEGbuffer + len, MT9D111_BUFFER_SIZE - len);    // initialize the PDC here (there is not enough time after VSYNC goes HIGH)
while(!(PIOA->PIO_PDSR & (1 << 15)));                         // wait for the start of a new frame (VSYNC LOW->HIGH)
while(PIOA->PIO_PDSR & (1 << 15));                            // wait for the end of the sampled frame (VSYNC HIGH->LOW)

MT9D111_register_write(WRITE_PAGE, 1);
uint16_t da5 = MT9D111_variable_read(9, 15, 1);               // last JPEG length [23:16]
uint16_t da6 = MT9D111_variable_read(9, 16, 0);               // last JPEG length [15:0]

Sampling the image on TT7F is a matter of configuring the Parallel Capture Mode and enabling the Peripheral DMA Controller associated with it. This is all done by calling PDC_transfer() function and passing it a buffer for the image and its size. Since the image is kept in SAM3S' SRAM the buffer has to be less than 64kB with sufficient space for the rest of the code to execute in. I generally use 45kB for the buffer. If the JPEG is larger, the DMA simply stops transferring the bytes. I then read variables ID-9 Offset-15 and Offset-16 which hold the size of the last image and I compare it to the size of my buffer. If it is larger I do the whole thing again. The process is timed by three while loops that wait for the right signals on VSYNC.

Hex	Marker	Description
FFD8	SOI	Start of Image.
FFE0	APP0	JPEG File Interchange Format.
FFDB	DQT	Define Quantization Table.
FFC0	SOF0	Baseline DCT.
FFC4	DHT	Define Huffman Table.
FFC4	DHT	Define Huffman Table.
FFC4	DHT	Define Huffman Table.
FFC4	DHT	Define Huffman Table.
FFDA	SOS	Start of Scan.
FFD9	EOI	End of Image.

The way I do it, I first create the JPEG header and place it in the big 45kB buffer. Then I pass a pointer to the end of it to the PDC function. In this way the image gets sampled right after the header where it is supposed to be. At the end I append an EOI marker (FFD9) and at that moment I have a complete JPEG image in a buffer including its length. The table above shows the structure of the resulting image in JPEG marker language. MT9D111 produces JFIF formatted images - the APP0 marker. The DQT marker then contains two quantization tables that were used in the compression. The SOF0 marker provides details about the scanned data (resolution, image components, etc.) and is followed by four huffman tables that contain the codes to decode the image data. The last marker before the actual image is SOS. It contains some more information about the image components and associated tables. EOI is just two bytes immediately following the last image byte denoting the end of the image.

The above is a screen of a hex dump of an example image with the markers highlighted. The five green bytes is the start of the actual image data.


Image Settings
With the basic image acquisition resolved it's time to look into some image settings. MT9D111 offers a number of drivers such as Auto Exposure, Flicker Detection, Auto White Balance, Histogram, Auto Focus or Forcing Outdoor White Balance. These drivers require only enabling in ID-1 Offset-2 variable. Each driver then has a variety of settings. I personally didn't tinker with their individual settings too much only tried enabling them separately and taking a number of images in different conditions. Based on that I decided to use Automatic White Balance and I also enabled Auto Focus even though I am not certain about its functionality as previously mentioned.

Another possibility to alter the output image is the Gamma correction in variable ID-7 Offset-68. Above is the default setting of Gamma = 0.45.

These were taken with the mid option of Gamma = 0.56.

And these images are without gamma correction (Gamma = 1.0). After comparing a number of images I leave the gamma setting at the default value of 0.45.

The 6th, 5th and 4th bit of the same variable (ID-7 Offset-68) set the Contrast correction. The first two images show the default setting called 'Noise reduction contrast'.

The second setting compresses dark and bright tones and stretches the mid tones based on an S-curve with slope of 1.25.

The third based on S-curve of slope 1.50.

And the fourth with slope of 1.75. The most usable setting to me seemed the second with 1.25.

JPEG Quality Factor represented by the Qscale variable (ID-9 Offset-10, 11 and 12) in MT9D111 context is another attribute to consider. Especially with its impact on image size. A comparison of quality factors of 50 and 90 is in the images above.

Image resolution, another thing to consider, is programmable via variable ID-9 Offset-7 and 9. The resolution of the example images in order is 320x240, 448x336 and 576x432.

quality	closet	view	horizon	sky
50	9206	6472	7577	6305
60	10400	7035	8567	7381
70	12239	8379	10328	7547
80	15074	10670	12930	8891
90	22931	16880	19777	15323

To put some numbers behind the decisions I made this and the following table. The first one shows the size of an image in bytes based on the quality factor it was taken with. All images had resolution of 320x240 pixels.

resolution	closet	frontyard	horizon	house
320x240	10073	11717	5414	7867
384x288	13994	16107	6880	11138
448x336	17891	20242	8604	14563
512x384	22646	26608	10344	18225
576x432	27047	32336	12400	22252
640x480	31793		14643	26673
704x528			17126	31407
768x576			19316

The second table contains the sizes of images in bytes differing in resolution while all taken with quality factor of 50.

Out of curiosity I wrote a script that tried to sample the image in parts using two buffers. One being fed with the data while the other one is used to push the already received bytes to PC via USB interface. The intention was to get the full resolution highest quality pictures MT9D111 could do. I succeeded only partially. I managed to stream JPEGs beyond the 64kB of RAM the MCU had, but at certain amount of data the transmission simply no longer kept up. The above is 800x600 pixel image with quality factor of 90 taking up 127,167 bytes.

This one is 1600x1200 pixels at a quality factor of 80 occupying 194,403 bytes. I had to clock the camera as slow as possible while fast enough so that the JPEG buffer used by MT9D111 wouldn't overflow. The resulting images also suffered from a lot of errors and required implementing restart interval of one (the compression would resynchronize after every Minimum Coded Unit) further increasing the size of the image.


Example
For initial testing I wrote this basic main() function that has the tracker take a picture, encode it and transmit the individual packets then repeat this over and over. The SSDV packets are interleaved with regular telemetry, the Ublox module is placed in a power saving mode, the MCU transitions between 64MHz and 12MHz main clock operation and the camera is put in standby while it is not being used.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153

// MAIN ------------------------------------------------------------------------------------------------------------------ int main(void) { PS_switch_FastRC_to_XTAL(); // 12MHz MCK EEFC_setup(0, 2, 0, 0); // Flash Wait State 2 + 1 (max. 64MHz), 128-bit access WATCHDOG_disable(); // disable WatchDog timer SysTick_delay_init(); // configure the delay timer for 64MHz MCK MT9D111_init(); LED_PA0_init(); LED_PB5_init(); ADC_init(); SysTick_delay_ms(188); // delay for the Ublox module to get ready, actual delay = delay * 5.333 UART1_init(); UBLOX_request_UBX(setNAVmode, 44, 10, UBLOX_parse_ACK); UBLOX_powersave_mode_init(setCyclicOperation_3s); UART1_deinit(); uint8_t Itry = 3; while (1) { PS_switch_MCK_to_FastRC(0, 0); // momentarily 4MHz MCK PS_SystemInit(31, 3, 1); // 64MHz MCK LED_PB5_blink(5); LED_PA0_blink(5); if(img_id % 5 == 0 && Itry) {res_x = 512; res_y = 384; Itry--;} // each 5th image will be higher resolution else {res_x = 320; res_y = 240; Itry = 3;} uint32_t imageLength = MT9D111_get_image(); // setup the camera, sample the image, return image length if(imageLength > 1) GPS_UBX_error_bitfield &= ~(3 << 0); // SSDV - image in the buffer else if(imageLength == 1) GPS_UBX_error_bitfield |= (1 << 1); // SSDV - communication with the camera error else GPS_UBX_error_bitfield |= (1 << 0); // SSDV - image buffer overflow PS_switch_MCK_to_FastRC(0, 0); // momentarily 4MHz MCK PS_switch_FastRC_to_XTAL(); // 12MHz MCK SPI_init(); SI4060_init(); SI4060_setup_pins(0x02, 0x04, 0x02, 0x02, 0x00, 0x00); SI4060_frequency_offset(0x00); SI4060_frequency(FREQUENCY_RTTY); SI4060_modulation(2, 1); SI4060_frequency_deviation(TX_DEVIATION_RTTY); SI4060_power_level(0x2A); SI4060_change_state(0x07); // TX state TC0_init_RTTY_NORMAL(); // setup the RTTY timer SysTick_delay_ms(1000); // time for synchronization - transmit the carrier (~5s) if(imageLength > 1) { uint8_t *_JPEGbufferP = JPEGbuffer; ssdv_t ssdv; ssdv_enc_init(&ssdv, SSDV_TYPE_NORMAL, SSDV_CALLSIGN, img_id++, 4); ssdv_enc_set_buffer(&ssdv, pkt); uint8_t c = 0; uint32_t packetCount = 0; while(1) { packetCount++; // Telemetry if(packetCount % packetsPerTelem == 0) // every x SSDV packets interleave with regular telemetry { telemCount++; UART1_init(); UBLOX_request_UBX(request0107, 8, 92, UBLOX_parse_0107); UART1_deinit(); ADC_start(); AD3data = ADC_sample(3, 100); AD9data = ADC_sample(9, 100); AD15data = ADC_sample_temperature(100); ADC_stop(); Si4060Temp = SI4060_get_temperature(); uint32_t telemLen = UBLOX_construct_telemetry_UBX(TXbuffer, 0); LED_PA0_blink(1); SI4060_tx_RTTY_string_TC0(TXbuffer, telemLen); // transmit the content of the buffer } // SSDV packet while((c = ssdv_enc_get_packet(&ssdv)) == SSDV_FEED_ME) { size_t r = 0; for(uint8_t i = 0; i < 64; i++) { if(_JPEGbufferP > JPEGbuffer + imageLength) continue; img[i] = *_JPEGbufferP++; r++; } if(r == 0) break; ssdv_enc_feed(&ssdv, img, r); } if(c == SSDV_EOI) { break; } else if(c != SSDV_OK) { break; } LED_PB5_blink(1); SI4060_tx_RTTY_string_TC0(pkt, 256); // transmit the content of the buffer } } else // in case the image acquisition fails transmit regular telemetry { LED_PB5_blink(100); LED_PA0_blink(100); telemCount++; UART1_init(); UBLOX_request_UBX(request0107, 8, 92, UBLOX_parse_0107); UART1_deinit(); ADC_start(); AD3data = ADC_sample(3, 100); AD9data = ADC_sample(9, 100); AD15data = ADC_sample_temperature(100); ADC_stop(); Si4060Temp = SI4060_get_temperature(); uint32_t telemLen = UBLOX_construct_telemetry_UBX(TXbuffer, 0); LED_PA0_blink(1); SI4060_tx_RTTY_string_TC0(TXbuffer, telemLen); // transmit the content of the buffer } TC0_stop(); SI4060_deinit(); SPI_deinit(); } }

The code first sets the MCU to 12MHz clock and does a few initializations. Noteworthy is the delay (14) before sending first commands to the Ublox module. It requires some time before it is capable of processing any. The delay itself lasts about a second despite being set to 188ms. The reason is that the SysTick routine expects a 64MHz MCK while here the code set it to only 12MHz. This should be taken into account at any other instance the MCU is running at speeds other than 64MHz. Upon entering the main while loop the MCU is set to maximum speed of 64MHz (25) and MT9D111_get_image() routine is called (33). Inside, the camera is brought from standby, reset and initialized to take a JPEG image in capture mode. If the Peripheral DMA Controller succeeds in sampling a whole image and the camera doesn't signal an error, the function returns the length of the image located in the JPEGbuffer and commands the camera into standby. Since the 64MHz clock is necessary mainly to sample the image, the MCU can slow down to 12MHz again (40) and save power. Following that the code initializes the transmitter (42-53) and for about 5 seconds transmits only the carrier (54) so the receivers can lock onto the signal. After that it checks the returned image length (56) and if there isn't any image it transmits only regular telemetry (124-147) and finishes the loop by de-initializing the transmitter (149-151). If there is an image, it does a few initializations of the SSDV routine (58-62) including setting the ssdv quality of the re-encoded image to 4. Consequently it enters a while loop which again every time checks whether to insert a regular telemetry string (72-92) in between SSDV packet transmissions. What follows is the main SSDV routine that byte after byte processes the image and fills an SSDV packet with the re-encoded data. When the packet is finished, it is sent to the transmitter to pass it onto radio waves (121). This repeats until the whole image is processed (110). Then the code exits the while loop and de-initializes the transmitter (149-151) completing one main loop.


Testing
To test the camera and the tracker in operation I used the example code from above.

The test setup consisted of one of TT7Fs, I had, powered with a 250mAh LiPo battery and my RTL-SDR dongle equipped with a simple 70cm ground plane antenna (switched to a yagi when testing at a greater distance). The tracker had just a quarter wave wire for an antenna. Throughout the test I moved the tracker a couple of times to capture different sceneries with different lighting.

The RTL-SDR dongle was operated by SDR# and the demodulated audio signal streamed via VB-Audio Cable to dl-fldigi. With the proper settings (RTTY, 300 baud, 800Hz shift, 8N2) dl-fldigi automatically recognizes SSDV packets and recreates the image from the received ones. If enabled, it also uploads them to ssdv.habhub.org.

The screenshot from tracker.habhub.org shows the interleaved telemetry working fine as well. A funny moment came about when a neighbour, not knowing what it was, picked the tracker up and took it home while it was taking pictures from farther away. The TT7S's detour is recoreded on the tracker website.

 

These are a few of the images produced by TT7F and received during the test. Unfortunately the weather didn't cooperate very much so most of the images are quite dark and cloudy. All the uploaded images can be accessed based on the callsign and date like this: ssdv.habhub.org/TT7S/2017-03-16.

I also measured the tracker's consumption in this setup. Using an Arduino DUE with its 12-bit ADCs I set up A7 to sample voltage on the battery (battery's positive terminal connected to the yellow lead). Since the DUE ran at 3.3V and a LiPo could be as high as 4.2V I had to bring the voltage down with a voltage divider (2x 100kΩ). The current was measured across a 1Ω shunt resistor placed between A4 and GND. The red lead connected to the tracker's ground while the black lead to the battery's ground. The tracker was powered from the yellow lead through its BATT+ terminal.

The DUE was programmed to sample every millisecond and average 250 values before sending the result to PC hence producing 4 averaged readings per second. The graph above shows about half an hour of the tracker running outside with good view of the sky. The blue curve represents the current readings, the smoother green line then 20 averaged current readings, and the last red curve is the battery voltage. The five spikes to 100ish mA region show the camera transitioning from standby to active and taking a picture before going back to low power standby mode again. The first two minutes of higher consumption were due to the Ublox module still being in acquisition mode - a state at which it actively searches for and acquires satellite signals. Later on with the GPS module transitioning to Power Optimized Tracking - a state of tracking, calculating position and staying idle - the tracker settled around 35mA during the transmission periods. The overall average consumption was then calculated to 36.91mA.

With this I have finished the second major block of code and functionality for TT7F (APRS being the first one). I intend to address power saving routines and testing the tracker's self-sustainability when running of solar panels in the future with a similar blog post.