PDA

View Full Version : firmware v.36



Phillip Kerman
2004-09-07, 11:08
I really want to be constructive so please take the following comments as
such. Do you have a list of features you test after each new build? That
is, if you fix a bug, can you then put the firmware/software through a
series of tasks that--as much as possible--explores all the core features?

Some software developers have such a testbed created that runs
automatically. That is, make a build... then run the test software to see
if anything new crops up.

Just an idea... you're probably doing something like this. It's just really
common for new bugs to get injected when one fixes another bug (and I mean
common to all software--not just slim).

Thanks,
Phillip

seanadams
2004-09-07, 12:15
Philip,

We have extensive automated tests for ethernet, ir, wireless, and audio
performance. However, it is extremely difficult to write automated
tests for user interfaces so we don't have a lot of automation there.

Before shipping any nightly I always do high-load testing with four or
five clients all streaming flac, mp3, and running both graphic and char
displays. There are quite a few permutations and mainly I am making
sure that all the hardware configs work the same. At home I run the
following:

SliMP3
Squeezbox wired, char display
Squeezebox wired, graphics display
Squeezebox wireless, char display
Squeezebox wireless, graphic display

Each of the different SB permutations is important wrt how the
low-level firmware drivers talk to all those different interfaces, so I
always have all of them running. The most important thing that always
gets a ton of testing is to make sure the updater works.

However not every nightly gets full regression testing of every feature
by a human - this is not the purpose of the nightlies, though as we
identify common issues they get added to a list of things that I do
specifically check.

A lot of the setup and IR code has been reworked in firmware v35 and
v36. These releases definitely fix some known crashers and that's why I
got them into CVS asap - a couple people needed those fixes right away.

The usual bleeding-edge warnings about nightly releases always apply.
We'll only point someone there if there's a specific issue that's been
fixed, not as a general rule. Do expect an occasional new bug (like the
report of static IP entry being broken). I am fixing that one right
now.

Sean

On Sep 7, 2004, at 11:08 AM, Phillip Kerman wrote:

> I really want to be constructive so please take the following comments
> as
> such. Do you have a list of features you test after each new build?
> That
> is, if you fix a bug, can you then put the firmware/software through a
> series of tasks that--as much as possible--explores all the core
> features?
>
> Some software developers have such a testbed created that runs
> automatically. That is, make a build... then run the test software to
> see
> if anything new crops up.
>
> Just an idea... you're probably doing something like this. It's just
> really
> common for new bugs to get injected when one fixes another bug (and I
> mean
> common to all software--not just slim).
>
> Thanks,
> Phillip
>
>

Michel Fombellida
2004-09-07, 13:09
Sean,

your fast reaction time is appreciated. Thanks!

Michel