s3.AttemptStrategy and the juju-core test suite

[Gustavo Niemeyer]

On Tue, Mar 12, 2013 at 3:15 AM, John Arbash Meinel
<john.meinel at canonical.com> wrote:
(...)
> a) Why do we have 2 implementations of AttemptStrategy
> (juju-core/trivial, goamz/ams) that are slightly different?

Because one of them was forked from the other and evolved to suit
needs. My goal is to have an even better AttemptStrategy in the near
future living in its own package. I've been solving real issues first.

> b) Should the attempt strategy actually just be a parameter to the
> s3/amz/ec2/goose/etc objects (rather than global state we have to
> override)? We could allow 'nil' to be passed, and then have it use the
> default timeouts we already have.

Doesn't sound like the right approach. AttemptStrategy is purposefully
an implementation detail, which has changed and will be changed again.
The goal is to have an API that tends to work, not one that needs
profound understanding of details and tweaking, nor one that will work
for someone in San Francisco and break for someone in Australia. If
there is a better way to do retrying than we're doing, let's integrate
that into the package itself.

> c) Should s3 itself be doing the retry rather than exposing the retry
> up to the higher level? [A gui using the library may want to give a
> quick update that the bucket doesn't exist even while it is doing a retry]

Yes, it should do the retry itself IMO. We shouldn't have to code
non-straightforward retry logic on every single application we code
against S3.

The API is still new, though, and in fact we'll likely have a v2 goamz
package in the near future to improve both the retrying in other cases
and the API itself.

I suggest the following approach to address the issue you raise:

1) For the short term, we can add a boolean setting to control
retrying on a specific S3 instance. When people are unhappy with the
internal retrying, they can build whatever they want on top of it.

2) For the medium term, we should be able to customize retrying or not
for specific method calls too.


gustavo @ http://niemeyer.net