Partial object updates
Object Storage has a mechanism for partial updating and appending of objects data in a bucket.
Note
The mechanism of partial object updates is not included in the S3 API standard functionality and is supported for buckets with versioning disabled.
With partial updates, you can store data (e.g., logs) in a bucket as a single file and append data to it from time to time.
This also simplifies large file processing. For example, to change one byte of information in a large file, you can use:
- Standard get and upload S3 API methods by fully downloading and uploading the object back to the storage.
- Patch method by uploading only the changed/new part of the file to the storage.
Such partial overwrite streamlines your Object Storage operations and reduces costs.
With partial overwrite, the following will be overwritten on the server side:
- Object as a whole, if initially uploaded using the PUT method.
- Object components falling within the update range, if the object was initially uploaded in parts.
This functionality is implemented as a patch method and is supported in GeeseFS.
GeeseFS is the recommended tool for partial updating of objects in a bucket.
You can also directly send patch requests to the Object Storage API, or extend the set of AWS SDK methods using the patch method specification.
Concurrent object updates
Object Storage supports concurrent partial object updates.
With a partial update, each parallel request is executed on a dedicated object snapshot. Therefore, different update requests for the same object are processed independently.
All changes during parallel requests are applied atomically.
Conflict resolution
A conflict resolution mechanism has been implemented for concurrent partial updates.
In a successful conflict resolution, the more recent update is applied.
The number of overwrite retries on the server side is limited. If the server fails to resolve conflicts, the user gets the ConcurrentUpdatesPatchConflict error.
Specification
"PatchObject":{
"name":"PatchObject",
"http":{
"method":"PATCH",
"requestUri":"/{Bucket}/{Key+}"
},
"input":{"shape":"PatchObjectRequest"},
"output":{"shape":"PatchObjectOutput"},
"httpChecksum":{
"requestAlgorithmMember":"ChecksumAlgorithm",
"requestChecksumRequired":false
}
},
"PatchAppendPartSize":{"type": "integer"},
"PatchedObjectInfo":{
"type":"structure",
"members":{
"ETag":{"shape":"ETag"},
"LastModified":{"shape":"LastModified"}
}
},
"PatchObjectOutput":{
"type":"structure",
"members":{
"Object":{"shape":"PatchedObjectInfo"}
}
},
"PatchObjectRequest":{
"type":"structure",
"required":[
"Bucket",
"Key",
"ContentRange"
],
"members":{
"Body":{
"shape":"Body",
"streaming":true
},
"Bucket":{
"shape":"BucketName",
"location":"uri",
"locationName":"Bucket"
},
"ContentLength":{
"shape":"ContentLength",
"location":"header",
"locationName":"Content-Length"
},
"ContentMD5":{
"shape":"ContentMD5",
"location":"header",
"locationName":"Content-MD5"
},
"ContentRange":{
"shape":"ContentRange",
"location":"header",
"locationName":"Content-Range"
},
"IfMatch":{
"shape":"IfMatch",
"location":"header",
"locationName":"If-Match"
},
"IfUnmodifiedSince":{
"shape":"IfUnmodifiedSince",
"location":"header",
"locationName":"If-Unmodified-Since"
},
"Key":{
"shape":"ObjectKey",
"location":"uri",
"locationName":"Key"
},
"PatchAppendPartSize":{
"shape":"PatchAppendPartSize",
"location":"header",
"locationName":"X-Yc-S3-Patch-Append-Part-Size"
}
},
"payload":"Body"
},