Fixing a Request Timeout Error When Uploading Files To Rackspace Cloud Files

We’re storing user uploaded files using Rackspace’s Cloud Files. We have a process that allows the user to upload to our server and then we “lazy” upload it to Rackspace. We do this because there have been a couple cases where we can’t cleanly upload them (due to a bug in our code) and having the local copy has made people feel better (we’re going to slowly roll this back once we determine we’re in good shape). As part of this process, we verify the file has been uploaded (making sure the file size in Cloud Files matches the file size on disk) and then we upload the file to a separate container in a different region so we have a backup (some of these files can’t be recreated so we’re being overly cautious).

This script that does the second upload looked like this.

$connection = $this->getConnection();

$count = 0;
do {
    $newFile = $this->findNewFile();

    if ($newFile) {
        $connection->uploadFile($newFile);
        unlink($newFile);
    }
    
    $count++;
} while($newFile and $count < 1000);

When we originally deployed the script it worked perfectly but as the number of files increased we started getting the following error randomly:

[27-Dec-2017 09:59:46 America/Detroit] Client error response
[status code] 408
[reason phrase] Request Timeout
[url] https://storage101.dfw1.clouddrive.com/v1/MossoCloudFS_<GUID>/path/to/file

[27-Dec-2017 09:59:46 America/Detroit] #0 /var/www/site/releases/20171227145101/vendor/guzzle/http/Guzzle/Http/Message/Request.php(145): Guzzle\Http\Exception\BadResponseException::factory(Object(Guzzle\Http\Message\EntityEnclosingRequest), Object(Guzzle\Http\Message\Response))

We weren’t able to find anything via Google that was helpful but after a lot of playing around we figure out it was actually a problem with the order we were doing things in. The call to findNewFile was taking several seconds to complete and this appears to be causing the timeout error. Our solution was to precalculate the list of files it was going to upload and now it works perfectly now.

$files = $this->getNewFiles();
$connection = $this->getConnection();

foreach ($files as $newFile)
    $connection->uploadFile($newFile);
    unlink($newFile);
}

It’s also a lot cleaner.