Turning a .byb recording into suspension numbers
An implementation spec. Given a .byb file and optionally a .bybLeverage curve, this
produces position, velocity, stroke and summary figures.
Steps are in dependency order. Everything is per suspension channel unless stated. Every rule
here has been checked against several recordings; where one does not hold universally it says
so at the point it appears.
1. Read the file
A 72-byte header, then fixed 48-byte little-endian records back to back, nominally one per
millisecond. Records start at offset 0x48. Treat the rate as nominal and always take timing
from the counter, not from the declared rate.
Header bytes 0 to 6 hold the recording's start time as single bytes: year minus 2000, a zero,
then month, day, hour, minute and second.
The mode string sits at 0x16, the sample rate at 0x20, and the sampling mode at 0x2A. All
three are null-padded text. Only the mode string is used downstream, to
select the sensor lengths below; the declared rate is deliberately ignored.
Each 48-byte record:
| offset | type | meaning |
|---|---|---|
| 0 | uint32 | sample counter, see below |
| 4 | uint16 | fork reading, 0 to 4095 |
| 6 | uint16 | shock reading, 0 to 4095 |
| 8 | 6 x int16 | accelerometer X/Y/Z then gyro X/Y/Z |
| 20, 22, 24 | uint16 | brake 1, brake 2, wheel speed |
| 26 | 2 bytes | unused |
| 28 | int32 | latitude x 10,000,000 |
| 32 | int32 | longitude x 10,000,000 |
| 36 | float32 | altitude in metres |
| 40 | float32 | GPS speed, already in km/h |
| 44 | uint32 | marker, always 0x0000FFFF |
Find the end of the data with the marker, not the counter. Take the first record whose
marker is not 0x0000FFFF, and keep that record plus everything before it. Everything after
is padding, and including it adds a stretch of fake stationary data that corrupts every
average.
Keep that last record even though its marker is not a marker. It fails the test because
the trailer described below begins inside it, overwriting those four bytes. Its first 44
bytes are ordinary data and nothing reads the rest, so keeping it is correct. Dropping it
instead shortens the recording by one sample and shifts several averages.
Do not stop at the first counter that fails to increment by one. Some hardware writes an
irregular counter that repeats and skips, such as 13, 14, 15, 15, 16, 17, 19, 19, 20, and
that rule can truncate a recording to a handful of samples.
Never assume a starting counter value; a selection export keeps the original numbering.
Always work from differences. The sample interval is likewise not always 1 ms, and a repeated
counter gives an interval of zero. Section 7 depends on this.
GPS values are held constant for 100 consecutive records, because the receiver updates at
10 Hz.
A text trailer holds the rider name, frame name, wheel size and the three travel figures, in
that order, as fixed-width null-padded ASCII fields.
Take the non-empty runs of printable characters and read the travel figures as the last
three, in millimetres: fork travel, shock stroke, wheel travel. Count from the end rather
than the start, because either name field can be empty and then there are only five runs, not
six. The name fields are whatever is left over. It begins 2012 bytes before the end of the file,
marked by a two-byte 0xF0F0 sentinel. That sentinel lands exactly on the marker field of
the record it overlaps, which is why the marker rule above finds the end of the data.
Seek to that fixed offset and check the sentinel is there; do not scan for the sentinel, since
those two bytes also occur inside ordinary record payloads. The three travel figures are
fork travel, shock stroke and wheel travel, in that order.
Sensor lengths are not in the file. They are properties of the hardware: 210 mm for the
fork sensor and 80 mm for the shock sensor on a mountain bike setup. You must supply them.
The accelerometer, gyro and brake columns are at the offsets above, but the conversion from
raw counts to g, degrees per second and percent is not covered here. Suspension work does
not need them.
2. Filter the raw readings
Apply to the raw 0 to 4095 values, in this order.
-
Median, window 6, biased toward zero. For each sample take the six values
x[i-3 .. i+2], three back and two forward, and sort them. Count how many are negative,
then pick as follows. More than three negative, take the fourth smallest. Exactly three
negative, return 0.0. Fewer than three negative, take the third smallest. Sensor
readings are never negative, so in this pipeline it is always the third smallest, but the
rule matters if you reuse the filter on a signed series.This is not a symmetric median of seven, and the difference is large. On an active
recording it changes roughly a third of the samples, by up to 2.5 percentage points at the
peaks; on a quiet one it changes fewer. Do not use the proportion as a self-check.Edges are handled on the output, not the input. Compute the interior for
i = 3 .. n-4inclusive, then set outputs 0, 1 and 2 equal to output 3, and the last
three equal to outputn-4. Note the interior stops atn-4, notn-3; that is why the
very last input sample never enters any window. If the series is shorter than seven
samples the filter returns nothing, and the caller uses the unfiltered series instead. - Moving average, width 5. Use the implementation in section 8.
- Flatten the head: replace the first 30 samples with the value at sample 30. Skip this
if the series has 30 samples or fewer.
3. Convert to millimetres and percent
Three details change the result. The [50, n-50] window ignores startup and shutdown
artefacts. Each sample is clamped at zero after subtracting the baseline and before
scaling, in sensor counts. And the division by 4095 happens before the multiplication by
the sensor length.
The subtraction is a pure shift removing the sensor's mounting offset. Never rescale to make
the maximum reach 100%.
travel is fork travel for the front and shock stroke for the rear.
4. Read the leverage curve
A .bybLeverage file holds shockStrokeMm,wheelTravelMm rows in millimetres, optionally
preceded by # comment lines carrying metadata. Points and metadata are read in two
independent passes over the same file.
The point pass
Read every line, top to bottom, and for each one:
- Trim whitespace. Skip the line if it is now empty.
- Split on comma, keeping empty parts. Semicolon and tab are not separators.
- Skip the line if it yields fewer than two parts. Parts beyond the second are ignored.
- Parse parts one and two as decimal doubles, and require the whole part to be consumed.
Reject hexadecimal, trailing units and anything with leftover characters, so5.0mmand
0x10both fail. Skip the line if either fails. - Append the pair.
There is no # test in this pass. Header lines survive it only because they contain no
comma, or fail to parse as numbers. Whitespace around the comma is fine, as are +5, .5,
5. and 1e2. Values like 5.0mm, 0x10 and 1,000 are rejected.
Points are kept in file order. They are never sorted, deduplicated, resampled or
snapped to a grid, and uneven spacing is allowed. There is no requirement that the table
start at 0,0, that it be monotonic, or that its last row match the declared travel.
Column one is shock stroke, column two is wheel travel. Both in millimetres.
The metadata pass
A curve file may carry # comment lines before the rows. Only two of the keys in them affect
any number in this document, so read only those:
Read from the top again. Trim each line and skip it if empty. If it starts with #, take
everything after the #, find the first colon, and split there into a key and a value,
trimming both. Key matching is case-insensitive. Stop at the first line that parses as
a valid data row, so # lines after the table are not read as metadata.
Both travel figures default to 0 when absent, which triggers the rule below. Other keys carry
descriptive information such as the curve's name, its brand and model, when it was created and
what units it claims. Nothing here consumes them, so ignore them unless you want them for
display.
Filling in missing travel
If either shockStrokeMm or wheelTravelMm is missing or not greater than zero, derive
both from the table as a per-column maximum, skipping any row with a negative or
not-a-number value:
This is a maximum, not the last row, so an out-of-order table still gives the right answer.
5. Map to the wheel
Keep two separate sets of lists, one at the shock and one at the wheel. Build the wheel
millimetre list in this order:
The smoothing comes after the mapping. Smoothing first, or skipping it, leaves the peak
fractionally too high and shifts every wheel velocity that follows.
With no curve
map is a multiplication by wheelTravel / shockStroke, both taken from the recording's
trailer, and both must be greater than zero. If that ratio is within 1e-12 of 1.0, skip the
mapping entirely and use the shock millimetres unchanged. With no curve file, percent at the
wheel also uses the recording's wheel travel.
With a curve
Fit a polynomial of degree pointCount - 1, so a 14-point curve gives degree 13. Lower
orders and straight-line interpolation both shift the wheel velocity extremes by around
2 mm/s.
Despite the degree, this does not interpolate the points exactly. Normal equations at
degree 13 over a 0 to 65 mm range are badly conditioned, and the fitted curve misses the
supplied points by up to about 0.02 mm. That residual is part of the specified result,
not a mistake. Do not substitute a better-conditioned solver to remove it.
Fit by normal equations with Gaussian elimination, in this exact form. Coefficients come
out in ascending power order, so c[0] + c[1]*x + c[2]*x^2 + ....
Evaluate with an ascending forward accumulation, not Horner, and with no clamping of
the input or the output:
The fused multiply-add is not optional. Substituting acc + c[k] * p rounds twice instead of
once and flips the sign of samples sitting within a bit of zero, which resizes the compression
and rebound buckets and moves the percentiles.
A general-purpose least-squares routine will not give this result. The coefficient order is
reversed from most libraries, and a well-conditioned solver scales the design matrix where
this method forms the raw normal equations.
Duplicate x values make the matrix singular and nothing checks for it.
Percent at the wheel is wheelMm / wheelTravel * 100, using the wheel travel from the curve
file rather than from the recording, though the two normally agree.
The mapped position never reaches exactly zero. At a shock position of zero the polynomial
returns its constant term, a fit residual of a fraction of a millimetre. That is expected.
6. Recording gaps and segments
Run this test on the raw counter differences in milliseconds, not on the seconds-based
time series used elsewhere.
Segment boundaries are {0}, then each gap index, then n. Filters, derivatives and stroke
detection must never run across a boundary.
A recording straight off the device has no gaps and yields a single segment, so this section
does nothing. It matters only for a file that has had stopped sections cut out of it and been
written back, which leaves real jumps in the counter. Without this handling a filter smooths
across the join and invents velocity spikes at the seam.
7. Velocity
-
Backward difference, written at the later index:
v[k] = (x[k] - x[k-1]) / (t[k] - t[k-1])in mm/s, forkfrom 1 upward. Setv[0] = 0,
and setv[k] = 0at every segment start and wherevert[k] <= t[k-1].Build the time series as
(counter[i] - counter[0]) / 1000in seconds, and divide by the
actual difference of two of those values. Do not substitute a nominal fixed step. - Despike. Walk
kfrom 1 ton-2. Ifv[k]falls below -5000 or above +10000 mm/s,
replace it with the mean of its two neighbours,0.5 * (v[k-1] + v[k+1]). The first and
last samples are never despiked. Work in place and forward, so a repaired value can be
read as the left neighbour of the next test. Ifk+1lies outside the current segment,
clampv[k]into the range instead of averaging. Normal riding never triggers any of this. - Moving average, width 11.
Wheel velocity is a fresh derivative of the wheel millimetre list, not the shock
velocity multiplied by a ratio. Take it after step 5.
Acceleration is the same three steps applied again to velocity.
8. The moving average, exactly
Width is forced odd with width | 1, so a request for 10 runs at 11 and 6 runs at 7.
half = width / 2. If the series is shorter than the width, return it unchanged.
The total is slid, not recomputed per window. Section 11 explains what that decides.
9. Strokes
Run on the millimetre travel list, not on velocity, one segment at a time.
Skip a segment shorter than 4 samples, or one whose values are all zero. Skip the leading
flat run by finding the first k >= 1 where seg[k] != seg[k-1].
Walk forward from there tracking a running extreme, alternating direction, starting by
looking for a maximum. Test reversal with these, where EPS = 2^-52:
When a reversal fires, record the index of the extreme value itself, not the current
sample, then flip direction and re-test the same sample without advancing. Leave the stored
extreme value alone at that moment; the re-test overwrites it.
When a sample differs from the running extreme by less than EPS, neither comparator fires;
advance and leave the extreme untouched. The extreme still standing when the walk ends is
not recorded, so the last turning point of a segment never becomes a stroke.
Initialise before the walk with the extreme set to seg[k+1] while its recorded index is
k, the direction set to "looking for a maximum", and the loop starting at i = k+1. That
asymmetry is deliberate; implement it literally. If k+1 is past the end of the segment,
the segment yields no strokes.
When the two lists come out the same length, drop nothing.
After the walk, collect the maxima and minima into two lists. If one is longer, drop its last
entry so the two match; if they are already equal, drop nothing. Then emit in exactly this
order, where c is the length of either list and mm is the millimetre travel list:
The loop stops at c-2, so the last maximum only appears as the end of a compression stroke.
Interleaving the two lists differently produces one extra rebound stroke. There is no minimum
duration filter.
A stroke is a start index, an end index and a direction, where 0 is compression and 1 is
rebound.
10. The two percentiles
Both are single elements, never interpolated between neighbours. n is the size of the
series being summarised, meaning the compression bucket or the rebound bucket for velocity,
not the total sample count. Evaluate the index as written, in floating point.
For an ascending series, sort ascending and return the element at k. For a descending
series such as rebound velocity, apply the same k to a descending sort, which lands on
signed index n - 1 - k. That is not the same as the 5th percentile of the signed
values: with 20 samples it returns the single most negative one.
11. What the running total decides
Section 13 splits velocity by strict sign, and samples that are exactly zero fall into
neither bucket. Section 8 decides how many samples that is.
While the bike is parked the position is flat, so every velocity is exactly zero. A moving
average that recomputes each window returns exactly zero and those samples are discarded. The
slid total instead stops changing on entering a flat stretch, because the arriving and
departing samples are equal, so it holds the rounding residue it carried in, around 1e-14.
The output is a tiny non-zero constant and every sample in the stretch is counted, with its
sign fixed by that residue.
How much this moves the averages depends entirely on how much of the recording is parked: on
one that is a third parked it shifts compression by about 28% and rebound by about 65%, and
on a mostly-parked one it can be a factor of three or more. Extremes, position figures and
everything stroke-based are unaffected either way. Implement section 8 exactly as written.
On a recording that is largely stationary, the compression and rebound averages and 95ths are
dominated by this residue and are not physical velocities. Treat them accordingly.
12. Which samples get counted
Statistics run over a list of index ranges, built by intersecting a requested window
[a, b) with a list of moving spans. A span is a start time and an end time in seconds.
Span times come from the GPS series, but the indices are into the full-rate sample list, so
search the per-sample time series:
If the span list is empty the result is the single range [a, b), the whole recording. If
a >= b the result is empty and the channel reports no data.
Note that a recording with no qualifying moving stretch at all also produces an empty span
list, and therefore covers the whole recording rather than none of it. If that distinction
matters to you, check whether any span was emitted before relying on the masked figures.
The same range set is used for position and for velocity. With no user selection, a is 0
and b is the sample count. With a selection, both come from binary searches into the time
list.
Excluding stationary time changes the answers substantially, and can move a position average
by several percentage points on a recording with a long pause in it.
Default to no masking, that is, an empty span list, which by the rule above gives the
single range [a, b) covering the whole recording. Build and apply spans only when a caller
explicitly asks for stationary time to be excluded. Every suspension figure in section 13 is
defined over whichever range set you supply.
One exception. Average GPS speed always uses the spans, whatever the caller asked for. It
is the only figure that does, and section 13 says so again where it is defined.
Building the spans
Work on the GPS series, which is one entry per 100 samples taken at indices 0, 100, 200
and so on, not on the full-rate data. A final partial block still contributes an entry, taken
like the others from its first sample rather than averaged.
The thresholds are 6 km/h and 2 seconds, with no edge padding. Treat them as fixed.
A configurable stop-removal setting with the same names exists elsewhere and is a different
feature; do not wire it to this.
Three details are easy to get wrong. The duration test measures from the first to the last
stopped entry, so a stop of exactly 20 entries spans 1.9 seconds and is kept as moving. Each
span ends at the time of its last entry rather than one step past it, so a span loses one
interval at its end. And the two-second comparison sits on a floating-point boundary: a stop
of exactly 21 entries computes as 2.000000000000001 rather than 2.0, so it qualifies. Compare
the subtracted times directly rather than rounding first.
Converted back through the range rule above, a recording whose speed first passes 6 km/h at
10.8 seconds and never stops again yields the single range [10800, 28701).
Record which convention produced a given set of figures. The same recording yields two
different sets depending on whether spans were applied, and the difference is large enough
to be mistaken for a change in the data.
13. Every figure on the summary page
Once sections 1 to 12 have produced a filtered position list, a wheel list, a velocity list
and a stroke list, the summary figures follow directly. travel is fork travel for the front
channel and shock stroke for the rear.
Timing
Duration comes from the counter values, not from the sample count. Those agree only when the
counter is regular. On a recording with an irregular counter, (sampleCount - 1) / 1000 is
wrong.
Moving time has nothing to do with the spans in section 12. It is a separate integration
over the GPS series, at one entry per 100 samples, with its own hard-coded threshold of
1 km/h, strictly greater than, and no minimum duration. Sum the interval to the next entry
for every entry that clears the threshold; the last entry has no next interval and so
contributes nothing. Stopped time is never measured, it is whatever is left over, which is
why it lands on values like 9.498 rather than a round tenth.
Do not use the section 12 thresholds here. They answer a different question and give a
noticeably shorter figure.
Ride
Take those three at the full sample rate across the whole recording, not over the GPS series
and not over the moving window.
GPS distance never touches latitude or longitude. It is the speed channel integrated over
time at the full sample rate, taking the speed at the later end of each interval:
Summing great-circle steps between fixes overshoots by roughly 1.7 times, because it
accumulates the receiver's positional noise.
Elevation gain and loss use a 1.5 metre hysteresis on the altitude channel at GPS rate,
one entry per 100 samples. Carry a reference level, and only when the altitude has moved
more than 1.5 metres from it do you bank that change and move the reference:
A plain sum of steps roughly doubles both figures, and a per-step deadband discards a slow
steady climb entirely.
Average GPS speed is the plain mean of the GPS series over the section 12 spans, built
and applied even when the caller asked for no masking. That includes the rule dropping all
spans when they total less than five seconds, which then falls back to the whole recording.
Include a GPS entry when its own sample index falls inside a range. Note this is the 6 km/h span rule, not the
1 km/h threshold that moving time uses. It is an unweighted mean of the entries, not weighted
by duration.
Position, per channel
Over the counted ranges from section 12, using the percent list and the millimetre list
separately:
Velocity, per channel
Split the velocity list by strict sign over the same ranges. v > 0 is compression,
v < 0 is rebound, and exact zeros fall into neither. Then:
Section 11 applies to compAvg, rebAvg, comp95 and reb95.
Strokes, per channel
Every figure in this block covers the whole recording. Strokes are never restricted to
the counted ranges, even when position and velocity are. Using the stroke list from
section 9:
totalStrokeM equals (avgCompLenMm * compCount + avgRebLenMm * rebCount) / 1000, which is
a useful cross-check on your stroke detection.
When a channel produces no strokes of a given direction, its average length is undefined.
Report it as such rather than as zero, which would read as a measured stroke of no length.
Both counts and both lengths therefore stay the same whether or not spans are applied, and so
does the stroke frequency, since its denominator is the whole-recording moving time.
Curve summary
For a leverage table, three figures describe its shape:
Label the curve Progressive when change < -3, Regressive when change > +3, and
Linear otherwise, or when the first ratio is within 1e-12 of zero.
Travel budget
Four percentages that sum to 100, describing how the recording distributed its travel. The
band edges are fixed constants, not derived from the data, and the input is the percent of
travel list over the counted ranges:
| band | range | label |
|---|---|---|
| 0 | below 10% | top-out |
| 1 | 10% up to 35% | sag |
| 2 | 35% up to 70% | mid-stroke |
| 3 | 70% and above | bottom-out |
Each band is half-open, so a sample exactly on an edge belongs to the upper band. Values are
not clamped, so anything above 100% still lands in band 3. Count the samples in each band,
then multiply by 100 before dividing by the total:
Multiply before dividing, as written. The four values will not sum to exactly 100, so do not
assert that they do.
Non-finite samples are not filtered out, and a not-a-number sample lands in band 3. Filter
them yourself if your input can contain any.
Two figures follow from the four percentages:
A negative index means the recording sat high in the travel.
Wheel versus shock
Every position, velocity and stroke figure above exists twice, once from the shock lists and
once from the wheel lists of section 5, and that includes the travel budget, which is a
position figure. Strokes are detected once, on the shock millimetre
list, so the counts and the stroke frequency are identical in both. Only the lengths and the
speeds change, and the lengths at the wheel are normalised by wheel travel.
Evaluate bottom-outs and deep strokes on each channel's own millimetre list against that
channel's own travel: the fork list against fork travel, the shock list against shock stroke.
For the rear, report the shock-based answer in both frames. Evaluating the same thresholds on
the wheel list gives a different result, because a progressive curve does not map 70% of
shock stroke to 70% of wheel travel.
Channels not covered
The accelerometer, gyro, brake and airtime figures need conversions from raw counts to g,
degrees per second, percent and metres that are not established here. Their record offsets
are in section 1.