`allocate_buckets` spending doesn't account for renewed leases.

Created by: tomprince

Currently, allocate_buckets only pays/charges for newly allocated buckets. However, in addition to allocating new buckets, any existing buckets for that storage index are renewed.

Created by: tomprince

https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1893 would remove the lease renewal from the operation.

Like we did for mutables, we could add an internal parameter that tells allocate_buckets not to do this and then use it from our implementation. I think this would make the behavior of allocate_buckets match what I thought the behavior was...

Of course, if we do that, we end up with shares in a bucket with different lease expiration times. Tahoe is ill equipped to deal with this. There is no API for renewing leases on a share, only on all shares on a bucket. So what do clients do? I guess they have to pay to renew leases on all shares in a bucket whenever any one lease gets uncomfortably close to expiration time.

I guess they have to pay to renew leases on all shares in a bucket whenever any one lease gets uncomfortably close to expiration time.

In #223 (closed) we are strongly considering doing almost this. However, it's not quite so bad. It's okay to have a lease close to expiration time as long as another lease on the same share file is not so close. All of the information to determine this is available to the server and our protocol even exposes it to the client (albeit with some bugs that need to be addressed).

Immutables are very likely to converge on synchronized leases very quickly because they just don't change that much (a bigger issue is that Tahoe makes it hard -- or impossible -- to allow some shares on a single server to expire which you might want to do if you've rebalanced some shares).

With #221 (closed) mutables might be a little worse off ... or maybe not. Tahoe's actual usage of mutables involves only updating all shares at the same time. So while the protocol allows a client to write to share N but not share M and our changes then may cause leases on shares N and M to diverge, the client implementation will actually only write to both shares N and M (and in fact all shares) for any update. A rebalancing problem similar to the one for immutables exists. It will probably need a similar solution (lease renewal that targets individual shares on specific servers, I guess? TBD).

I submitted a Tahoe-LAFS PR - https://github.com/tahoe-lafs/tahoe-lafs/pull/1160 - that allows us to control the implicit lease renewal behavior of allocate_buckets.

We will use this to just turn the behavior off.