Mechanism: only EnvironmentErrors are being handled nowadays to
trigger the error-message-accumulation and reporting logic, so fake up
an EnvironmentError as to let normal error handling take place here.

This is definitely a kludge, but it doesn't seem worth it perform the
obvious alternative option, which is making a new exception type to
treat the same problem.

Signed-off-by: Daniel Farina <daniel@heroku.com>

For ./setup.py in the shell

Signed-off-by: Daniel Farina <daniel@heroku.com>

This has the main benefit of being conceptually cleaner (cleaning up
the tar_partition module) as well as thought to being much faster in
cases where there are many files being archived.

As a target of opportunity, refactor to clean up complexity in dealing
with optional pipeline steps, by making PipelineCommand lazy (init,
then start).  There is still some complexity because one must call
.start() before pushing around file descriptors, since they do not
have a defined value before the program in question is started.

Signed-off-by: Daniel Farina <daniel@heroku.com>

An optional 'structured' keyword argument taking a dictionary is now
supported and logged (and is *not* escaped) in the key=value<SPC>
format.

Signed-off-by: Daniel Farina <daniel@heroku.com>

That was a lot of flakes.  However, there were also some false
positives that I gave treatment, too.

Signed-off-by: Daniel Farina <daniel@heroku.com>

Signed-off-by: Daniel Farina <daniel@heroku.com>

I tried to find a way to write a test for this, but it led to a lot of
overwrought code for this error, which I have witnessed but once ever:
in particular, testing the retry behavior (inverting control seemed
invasive and made the code harder to read).

Signed-off-by: Daniel Farina <daniel@heroku.com>

Signed-off-by: Daniel Farina <daniel@heroku.com>

Signed-off-by: Daniel Farina <daniel@heroku.com>

It doesn't look like any storage format changes will be happening in
the next release, and as-is this breaks inter-operation with existing
0.5.x archives.

Signed-off-by: Daniel Farina <daniel@heroku.com>

This was raised by Ed Muller <edward@heroku.com> as something that
makes the structured format much more useful.  Unwilling to duplicate
the time and pid information at both the log header and in the
structured information fields, this patch opts to move time to the
structured area.

However, as a service to human readers, the time and pid fields are
guaranteed to come first.  Previously, all fields were lexically
sorted.

Signed-off-by: Daniel Farina <daniel@heroku.com>