Moving to new site and reader summary after 66K

Recently I set up a proper website for the HAPP project and I'm transitioning the blog over to the new host at Squarespace. Come check it out!

As a final sign-off from this location at Google's Blogger, I'd like to give a geographical breakdown of my readers here over the 22 months that this blog has existed (22 months to the day in fact!)

I've had a total of 66,587 visitors. As usual, most visitors are from the USA. The last time I posted stats we were at 35,000 visitors, and the #2 location was France. This time Canada steps up to join its North American neighbor. O Canada! The UK only lags by a rounding error.

After 22 months and 66,587 visitors

The top browsers used by readers are Chrome and Mobile Safari. The top posts are Controls: Implementation and Valves and Pneumatics.

Sayonara from Blogger and see you at the new site!

Flight testing: HAPP-T1 and HAPP-T2

I still have a few posts to write regarding the HAPP's systems and construction; for example, the sensor and camera suites, and the ground side of the satellite communication system. Nevertheless, I thought I'd catch you up on the first two test missions, HAPP-T1 and HAPP-T2.

Neither one involved free flight. In fact, HAPP-T1 never left the ground. Let's step through the purpose of each test and the results.

HAPP-T1 was a live test of satellite telemetry in the field. Basically, I sat the HAPP core sans aeroshell in a grassy snowy area with good sky exposure, and we verified (1) downlink of GPS positional data and spacecraft status, and (2) uplink of commands. James monitored the server-side action from mission control in Austin (sadly, not Houston) while I handled ground operations in Ann Arbor. The HAPP ran exclusively on internal power.

Is this an Imperial Probe Droid on
Hoth or the HAPP core in a snowy field?

HAPP-T1 was a resounding success and produced data as shown in the image below. If you look closely at the numbers labeled "Payload" you can see the GPS coordinates being received by mission control. The red pin on the map is the location. The data payload from the satellite also contains various status codes and information such as vertical velocity and heading. We'll cover that in a future post.

Mission HAPP-T1: Success

So, onward to mission HAPP-T2. The goal of this mission was to shake down the balloon system and perform a tethered flight, thereby demonstrating all the functionality of a full mission while constraining the HAPP below a certain fixed altitude, in this case about 250 meters. Everything about this mission was live: Ground crew operations and staging, balloon inflation procedures, HAPP assembly and startup procedures, launch, ascent, communications, video, cutdown from the balloon, parachute deployment, and recovery.

This was my first time to inflate a large balloon, and fortunately I had an outstanding group of ground crew volunteers helping to wrangle the thing. Thanks guys!

I'll let the mission video speak for itself:

OK so what just happened? 

Problem 1:

Well, obviously we didn't fly very high, and therefore we didn't get to test the Earth Landing System, a.k.a the parachutes. Everything worked great up through launch. The HAPP started to ascend, and then at an altitude of about 3 or 4 meters there was a gunshot sound - the cutdown pyros severed the connection between the balloon and the HAPP. The Flight Controller computer decided to abort the mission!

I had to get home and check the computer log to understand what happened. It turns out that the FC performed correctly. The software has a geofencing feature that terminates the mission if the HAPP drifts out of a pre-defined geographical zone. I forgot to check the software settings, and the latitude geofence was set to 42.398 degrees north, which is a bit north of Kalamazoo, Michigan - a value chosen earlier in development for no particular reason other than it looked a bit farther north than my home town of Ann Arbor. Who knew that our launch site for HAPP-T2 at a park near Ann Arbor was at 42.399 degrees north? As soon as the HAPP became airborne, the FC detected a geofence violation and aborted the flight. 

This issue won't reoccur as I have now modified the FC sotfware to perform a geofence check during prelaunch procedures. If we're outside the geofence, the FC will go into an error mode and won't let us launch.

Although we were unable to witness the glory of the HAPP descending under its three huge parachutes à la Apollo, we were able to confirm almost everything else (Note: As an homage to NASA, the HAPP chutes utilize the Apollo red and white color scheme). In particular, the calculations were spot on for the amount of helium required to achieve the desired lift. The balloon harness and tether system also performed well.

One thing I could not confirm were flight dynamics. The mission software was configured to let the HAPP fall from 250 meters altitude for a few seconds before blowing the chutes. Inertial data from the HAPP's IMU could then be used after the flight to calculate acceleration, which in turn enables calculation of the aerodynamic drag coefficient or Cd. Knowing Cd, I can predict various things like the freefall speeds at different altitudes and anticipated shock forces when parachutes open. I had also hoped to confirm aerodynamic stability - see my old issues with the center of gravity. Those issues are solved on paper (well, in the CAD model at least) but I'd like to confirm with a physical test.

Problem 2:

When the FC aborted the mission, it also should have blown the parachutes. The chutes would not have done much good from an altitude of 3 meters - no time to inflate before ground impact - but they should have blown nonetheless.

It turns out that the FC did fire the parachute pyros. However, the steel ram pin that's supposed to puncture the liquid CO2 canister did not fully penetrate the canister. This is the first failure I have had of the ELS, including all of the early prototypes. This system has been highly reliable. I'm not sure what happened but I suspect I loaded the pyros with a short charge of black powder. I'll check carefully in the future. I'll also swap out the fire pin for a new one with a sharper tip. You can see the pin if you zoom in on the center of this photo; it's centered inside the steel spring.

Problem 2 is slightly concerning to me, but given the past reliability of the system, I'm not going to lose sleep over it. Not to mention, I calculate the terminal free fall velocity at low altitude without chutes to be around 40 kph / 25 mph, so the HAPP will likely survive a total chute failure. Heck, in HAPP-T2 the ground impact speed was around 28 kph, and there was almost no damage. By design, the HAPP is essentially a big, hollow, flying wing.

Overall, I judge HAPP-T2 to be an 80% success.

The Good News:

The HAPP structure performed in an exemplary manner during HAPP-T2 and the subsequent ground impact. My original design spec called for crash resistance from a free fall drop height of at least 1 meter onto a hard surface. In T2 the HAPP fell from 3-4 meters onto hard packed dirt. The aeroshell flexed as designed and did not suffer any damage. The internal structure and custom-molded crash pad also performed well with only slight damage to the structure in two locations.

Location 1 was the bottom of the structure where the tank deck is bonded to the central strut with specialty epoxy and small L-brackets, all made of carbon fiber (see my previous post for details about the construction). The epoxy bonds fractured before the carbon fiber components - as intended - and in this picture you can see my fingers in the resulting gap between the brackets and the now free-floating tank deck. A little fresh epoxy in the gap and the repair was complete.

Had the tank deck been fixed to the brackets with bolts, one of the carbon fiber components almost surely would have failed, and the repair would have been quite expensive, possibly requiring CNC machining of a new tank deck (see video of machining in previous post). Bonding was a good choice.

Damage location 1

Damage location 2 was the joint between one of the aerodynamic strakes and the ELS deck, as shown below. In the picture I've gone ahead and snapped the remaining good bond between the strake and central strut so I can chip away all the old epoxy and make clean new bonds.

Damage location 2

This repair was also pretty easy, but I had to break out my old assembly jigs to ensure the strake was aligned correctly when bonding. As described in a previous post, I 3D-printed various assembly jigs way back when I first built this version of the structure - glad I kept them around! 

3D-printed jigs held in place with clamps;
very useful for alignment when bonding
carbon fiber pieces with epoxy

OK, this post was a bit lengthy, but hopefully you enjoyed the walk through missions HAPP-T1 and T2. I was hoping to avoid a T3 - see the previous post describing costs for the balloons and helium! - but I'm going to repeat the last mission. I simply don't want to release the HAPP for a full mission without validating the parachutes and flight dynamics.

Stay tuned - we're just waiting on a break from the crappy weather during a typical Michigan winter!

Balloon system: Harness and tether

In the previous post I talked about balloon technology. Here I'll describe how we attach the HAPP to the balloon. To set the stage, recall that we are attaching a 10 kilogram / 22 pound load to a piece of thinly stretched latex rubber and we require it to survive turbulent winds and the like as it ascends to the target altitude of 30 kilometers / 100,000 feet.

My finished solution looks like this, with the balloon neck looped through a steel ring and clamped off with carbon fiber plates and steel screws:

Hang loose, dude

A quick search of other people's hobby balloon projects reveals the "standard" method of tying off using rubber bands and electrical tape. Follow the link and check out the pictures. This solution is not going to be capable of handling our loads; we need something more robust.

My idea was to use the steel ring in lieu of a loop of string formed from the tether itself. This prevents the abrasive Kevlar string from cutting through the latex. And to resist the pull-out force, I clamped the balloon neck back over onto itself using hand-cut carbon fiber (scraps from construction of the HAPP structure!) as pressure plates.

To enhance the grip of the plates onto the rubber while avoiding sharp, lacerating edges, I lightly scored the inner mating surfaces of the plates with a Dremel rotary tool. See frame (3) in the composite image below. After tightening the screws, this baby is going nowhere. I haven't actually done the test, but I'm pretty sure the rubber will shred before the steel ring slips out of this assembly.

Clockwise from top right:
(1) Balloon neck, (2) Neck being folded,
(3) Serrated inner surface of carbon fiber plate,
(4) Assembled harness with steel ring

One advantage of using the steel ring is that we can accommodate two different setups quite easily. The main setup is, of course, for flying the HAPP, with a single tether line attached to the craft. See the left frame below. You can also see a small generic fishing swivel I spliced in-line to decouple any rotations of the balloon and avoid transmitting them to the HAPP. No sense in making the stabilization jets work any harder than they need to - we do have a limited supply of gas! This swivel has a rating of 600 pounds which is almost 30X the static system weight, providing a comfortable safety factor even under extreme dynamic loading.

The alternative setup is for low-altitude test flights where the system remains tethered to the ground while still allowing the HAPP to hang freely below the balloon. See the right frame. In a tethered test we have three long lines running to tie-downs on the ground to prevent the balloon from moving around in any winds. See the video of the first test flight. I used three of these large kite reels to stow the ground lines.

In both cases we can use small screw-lock caribiners to connect to the steel ring for quick assembly / disassembly during testing. On flight day we can eliminate these carabiners and simply tie the single main tether directly to the steel ring for additional weight savings.

Left pane: Normal flight configuration
Right pane: With 3 ground tethers for test flights

What about all those knots tied in the kevlar lines? They're technically hitch knots. I like the Grapple Hitch for a variety of reasons, including its security under load or not, its adjustability, and the ease of untying.

Finally, let's look at the attachment to the HAPP itself. The main tether is fairly long at 5 meters. Why 5? I chose this length to achieve a nice slow natural oscillation of the HAPP. From the formula for a simple pendulum you can show that the period of back-and-forth swinging will be about 4.5 seconds (the reality is a bit more complex as this system is actually a compound pendulum - you can verify the details :-).

The thether is tied off to the machined aluminum tether ring on the HAPP. In-line with the tether are the doubly-redundant cutdown pyros. I also tied a small opaque cap to the tether. This cap is placed over a small photoresistor that's affixed to the HAPP. During normal flight the photoresistor does not sense any light and we can be sure the HAPP is still attached to the tether. When the pyros fire and the HAPP falls away, we will get a positive signal that separation is complete. More on this in a future post discussing the HAPP's sensor suite.

String A ties to the HAPP tether ring in the
right frame. Cap B goes over a photoresistor
which is not yet present at B in the right pane.
Blue cylinders are the cutdown pyros.

OK, that's it for the harness and tether. Thanks for hanging around!

Balloon system: Balloon and helium

The last few posts have covered electronics. In this post I'll go back to the mechanical side and describe exactly how we use a weather balloon to fly a 10 kilogram / 22 pound “spacecraft” off into the great blue yonder.

First let's discuss the balloon. Weather balloons are typically made from natural latex rubber. Literally, this is the material they extract from trees by pounding spigots into the trunks with hammers. While you might think there are suitable synthetic alternatives, the reality is that natural rubber has some unique performance properties, in particular its hyperelasticity - good for a stretchy balloon.

Balloons come in a variety of standard sizes which are categorized by the mass of rubber used to form the balloon, such as 100 grams, 600 grams, 1000 grams, and so on. Which size to use? The engineering trade-offs basically entail balancing four things: How high we want to go, how fast we want to ascend, ensuring the balloon eventually bursts from expansion in thin air, and the weight (and cost) of lifting a heavier balloon.

For the HAPP, we want to lift 10 kilograms to 30,000 meters / 100,000 feet, so we need 10 kilograms of lift, right? Nope. The balloon itself has mass, as does the bridle and tether connecting the HAPP to the balloon. Furthermore, we don't want to merely balance all the gross weight with lift; we need some net positive lift to ensure we ascend at a reasonable rate. Too fast and the balloon will generate turbulence behind it as it rams through the air, leading to a very bouncy flight - the opposite of what we're trying to achieve with a jet stabilized photography platform! Too slow and the HAPP will drift far downrange in the wind before it reaches the target altitude (and the onboard batteries may expire as well). Based on experience from other balloonists, it appears that the sweet spot is about 3 to 4 meters per second of vertical velocity.

Of course I verified the math by hand, but you can use a variety of online calculators to show that we need about 13,000 liters / 460 cubic feet of helium at standard temperature and pressure to provide the requisite lift. Using the 139 cubic foot aluminum pressure tanks that my local welding supply store uses for helium, this equates to about three and a half tanks. It also equates to a sphere of helium about 3 meters / 9 feet in diameter at STP, just like this:

1200 gram balloon inflated with about
13,000 liters / 460 cubic feet of helium
during the first HAPP test flight

A hint if you wish to try the calculations yourself: The ideal gas law and Archimedes' Principle are your friend, as is knowledge of the relative molecular weights of helium and air (mostly nitrogen). And don't forget about the coefficient of drag on the balloon. Bonus points if you show your work :-)

Three of the four 139 cubit foot tanks
provided by my local gas supplier, loaded
into my SUV so you can gauge the size.
Not tiny, but they're aluminum and light.

If we use a tiny balloon, say 200 grams, it will burst before it stretches to 3 meters in diameter as we inflate it on the ground. If we use any size other than the smallest balloons, say 600 grams, we can inject sufficient helium to achieve liftoff. However, there won't be much room left for the balloon to stretch as it expands in the thin air at high altitude. It will burst before reaching our target altitude.

So we should just use a ginormous balloon and be done with it, right? Nope again. An excessively large balloon may not burst before it stops rising, resulting in what is affectionately known as a floater. Floaters are bad because they can do things like circumnavigate the globe before the latex finally degrades in the ultraviolet rays of sunlight. Note that the HAPP guards against this possibility by using doubly redundant pyrotechnic devices to sever the balloon tether after time and / or distance limits have been exceeded. At that point the payload is gone and the balloon will resume ascending until it eventually bursts (and even if it didn't, the North Koreans would likely be less upset with a simple balloon floating overhead as opposed to a high-tech capsule laden with cameras and satellite communication devices!)

Fortunately, the largest balloon size easily available for general purpose, 3000 grams, appears to be about right. That's what we'll use for the main mission. They're also quite expensive, almost $400 apiece. Here are the two I've ordered:

Two 3000 gram latex weather balloons
from the good folks at  Scientific Sales

To conserve funds, we'll use smaller ones such as this 1200 gram balloon for initial flight testing (still $115 each!).

While we're discussing money, I should also note that a 139 cubic foot cylinder of helium costs $144 to fill where I live, so four cylinders ring up at $576. That means a full mission flight will cost almost $1000 for the balloon and helium alone, with any test flights, hopefully only one or two, costing somewhat less. Crikey! Well, in for a penny, in for a pound I suppose...

OK, with the balloon selection and helium calculations out of the way, in the next post I'll talk about attaching the HAPP to the balloon.

Onward and upward!

Electronics: Flight Controller Computer (FC)

And now to explain the Flight Controller (FC) computer, here’s the first guest post on this blog featuring James, the mastermind behind the FC. James is a friend from high school who now lives in Austin and has joined the HAPP campaign during the past several months as we attempt to bring this project to fruition. James is also a “real” software developer, not a pretender like me, and not only has he expanded what I thought possible on this project, he’s also accelerated the whole project significantly.

I'll let James take it from here!

==============================

In the last HAPP blog update, Chris provided details about the Autopilot (AP). This post will provide some background on the Flight Controller (FC), the companion computer onboard the craft.

To recap from the system architecture overview, the FC is responsible for several key functions:

• Determining GPS coordinates, altitude, speed (horizontal and vertical), and heading
• Transmit GPS data plus other flight data of interest to the ground via satellite
• Receive ground commands, if needed, via satellite (normally the HAPP is autonomous)
• Process all flight data and commands and decide what actions to take, such as firing pyros to cut the umbilical with the balloon or pyros to deploy the parachutes
• Monitor the AP status and abort the mission if the AP stops responding normally (recall that the AP is listening similarly to the FC and can also terminate the mission independently if needed)

Electronics:

Like the AP, the FC is an Arduino Mega with two custom shields. There were more than a few revisions prior to the delivery of final product for the launch. My professional background resides entirely in the software realm. I’ve written C-based firmware for embedded devices but I’ve never done much of the electronics work myself. This project was a great bit of fun and provided me a chance to stock up a full lab of gear (a desoldering gun is essential) and components for future projects as well.

I began with the usual breadboard prototype, then moved it to a more solid structure (protoboard) that required some light soldering. During this breadboarding phase, I worked out the beginnings of the FC Arduino Sketch that would serve as the scaffolding for the FC logic. I also mocked up a dummy AP Sketch that did nothing but communicate on the I2C bus.  You could type in state transitions manually and see the status codes flow to the FC. Once satisfied that I understood how each peripheral would work, I spent a considerable amount of time on electronics revisions. Each involved multiple SKUs of the same component from manufacturers. Some with built-in antenna, some without. Some with different form factors, others with different voltage requirements. We finally settled on the smallest available of each component that would suit our needs.

• GPS: u-blox MAX-M8Q
• Satellite: RockBLOCK 9603; uses the Iridium satellite constellation
• microSD: Adafruit MicroSD breakout
• Antennae: External Helical w/SMA (Satellite), Active PatchAntenna w/SMA (GPS)

The first unit is still in use and is my go-to development unit here in Austin. It features built-in antennae for GPS and satellite communications and is very portable, which is good when you have to sit outdoors to test satellite communication. The craft-ready revision has small coaxial connectors (SMA) for external antennae. Those antennae sit on the upper ELS deck, which is exposed to the sky, thereby avoiding the Faraday cage effect caused by the carbon fiber shell of the craft.

 

Unit #1 for development purposes.
Unit #2, which is on the HAPP, has a cleaner layout.

Unit #2 installed on the HAPP
as flight hardware

Autopilot (AP) on the left and
Flight Controller (FC) on the right,
as installed on the Electronics Deck

The electronics of the FC are much simpler when compared to the AP.  We have a single I2C device to communicate with – the AP.  We have two serial devices (GPS, Satellite) and a single SPI device (microSD).  The Arduino Mega is equipped with four serial ports, three of which are available via the Arduino header pins where we have access to the transmit (TX) and receive (RX) pins of each (the first serial port is accessible through the onboard USB connector and is used to program the Arduino and serves as a debug port). 

Future electronics enhancements might include the addition of another temperature sensor for redundancy (the AP has one) as well as a traditional GSM cellular communication channel for more frequent and higher volumes of data while the craft is at low altitude. This could aid in recovery efforts should we lose satellite communication.  Chris added this commercial, off-the-shelf tracking device from SPOT as a backup in the meantime.

Software:

The FC sketch has an initialization phase where it confirms that the peripherals are in working order. It opens two log files on the microSD card, one for general log data and another for CSV-formatted GPS data for later viewing in a tool of your choice.

Once setup succeeds and the FC reaches a state called PRELAUNCH_AWAIT_FCAP_HANDSHAKE, we await communication with the AP before handing off flight control to the FC logic.

The main microcontroller loop of the FC code services the GPS peripheral. For simplicity, we chose to stick with a serial interface for our chosen chip, the MAX-M8Q from u-blox. I found a high-altitude capable variant mounted to a breakout board by Uputronics out of the UK. Note that most civilian receivers will cease functioning around 18,000 meters / 60,000 feet. To track the HAPP up to 30,000 meters / 100,000 feet, selection of the right GPS receiver is critical.

GPS chips are constantly sending data to the communication bus. If you don’t devote time to servicing that stream of data, you risk losing characters and your GPS location fixes are in jeopardy. The cadence of the fix message is approximately one per second – more than enough for our positioning needs as GPS data is not involved in any kind of real-time steering of the craft. There is another class of GPS chips that provides higher accuracy and higher fix frequency. Coupled with predictive algorithms, that’s what your Tesla or military drones use for navigational guidance.

While GPS fixes are being assembled, the FC sketch yields time to a common worker function.  It is in that worker function where the FC does its work.  That outline is:

• Ask the AP for its state
• Sync onboard clock with GPS satellite time
• Log positional and operational data
• Evaluate states for flight control logic (the secret sauce)
• Repeat ...

Chris wrote logic that reacts to the navigational input from the fix data.  He reacts to flight data related to positioning, velocity and heading. He can respond to questions like

• Are we well outside of the bounds of the expected flight plan? Yes? ABORT! (There is a good story here from the first flight test. I’ll let Chris post about that later...)
• Are we descending unexpectedly? Yes? ABORT!
• Did we go supersonic on descent? Yes? Cool! Tell us about it.
• Did we lose contact with the AP? Yes? ABORT!
• More …

I should note here that the FC has multiple digital output pins available via a header soldered to top Arduino shield.  This is similar to an identical one on the AP.  The redundant pins allow either the AP or the FC to issue critical abort control messages to the craft – fire the pyros to cute down the balloon, release the chutes, etc.

Our satellite modem is able to send messages approximately every 30 seconds.  So, while we are constantly collecting flight data, we can only send data to our application server twice per minute. Once the FC logic detects that it is time to tell the ground (that’s us!) about the craft’s current state, it assembles a formatted payload and hands it off to the modem where some internal retry logic then attempts to acquire a satellite signal and send the data. These messages are called Mobile Originated messages (MO). It is also at this point where the modem can tell the craft about any pending commands sent from the ground. These are called mobile terminated (MT) messages. At present, we only issue a single command which tells the craft to enter the EXECUTE_ABORT state immediately – think BIG RED BUTTON.  We did, however, design it to be a general-purpose command control feature where we can set the AP or the FC to any state desired.

While the satellite modem is attempting to communicate, it hogs the microcontroller’s time awaiting a response. That is bad because we need to service the incoming GPS data and we need to service our own operational flight logic. To solve that, the library we use to talk to the modem allows us to provide a callback function so that it can yield time back to us. It is the same exact function that our GPS loop uses to process normal flight data.  So now we have two entry points into that single function – our GPS working loop and the callback from the satellite library. That can be bad.  Such code is said to be re-entrant since there are multiple processes or process threads accessing the function at the same time. It doesn’t have to be bad, but our satellite library allocates memory in such a way that we’d be clobbering variables set by the other callers. To solve that, our sketch uses a semaphore to gate access to the non-reentrant parts of the library.

During nominal operation, the FC will remain busy getting a GPS fix, providing navigation data to the flight logic handler, logging data to the microSD, sending data to ground control, and receiving messages from ground. It does this in a tight, highly-optimized loop from mission start to finish.

In an upcoming post, I will detail the software we call the HAPP application server.  We have servers in the cloud (because of course) where we coordinate all of the messages and present data logically (missions, messages, craft, etc.) for ground crew and observers.  It has a web site as well as an API for use by a dedicated mobile application currently in development by some amazing volunteers at Menlo Innovations in Ann Arbor, Michigan.