Partial object updates
Object Storage supports partial updating and appending of objects data in a bucket.
Note
Basic S3 API operations do not include partial object update. This feature is supported for buckets with versioning disabled.
With partial updates, you can store data (e.g., logs) in a bucket as a single file and append data to it from time to time.
This also simplifies large file processing. For example, to change one byte of information in a large file, you can use:
- Standard GET and UPLOAD S3 API methods by fully downloading and uploading the object back to the storage.
- PATCH method by uploading only the changed/new part of the file to the storage.
Such partial overwrite streamlines your Object Storage operations and reduces costs.
With partial updates, the following will be overwritten on the server side:
- Object as a whole, if initially uploaded using the PUT method.
- Object components being updated, if the object was initially uploaded in parts.
This feature is implemented as a PATCH method and is supported in GeeseFS.
GeeseFS is the recommended tool for partial updates of objects in a bucket.
You can also directly send patch requests to the Object Storage API, or extend the set of AWS SDK methods using the patch method specification.
Concurrent object updates
Object Storage supports concurrent partial object updates.
With a partial update, each parallel request runs on a dedicated object snapshot. Therefore, different update requests for the same object are processed independently.
All changes during parallel requests are applied atomically.
Conflict resolution
A conflict resolution feature is available for concurrent partial updates.
In case fo success, the more recent update is applied.
The number of overwrite retries on the server side is limited. If the server fails to resolve conflicts, the user gets the ConcurrentUpdatesPatchConflict error.
Specification
"PatchObject":{
"name":"PatchObject",
"http":{
"method":"PATCH",
"requestUri":"/{Bucket}/{Key+}"
},
"input":{"shape":"PatchObjectRequest"},
"output":{"shape":"PatchObjectOutput"},
"httpChecksum":{
"requestAlgorithmMember":"ChecksumAlgorithm",
"requestChecksumRequired":false
}
},
"PatchAppendPartSize":{"type": "integer"},
"PatchedObjectInfo":{
"type":"structure",
"members":{
"ETag":{"shape":"ETag"},
"LastModified":{"shape":"LastModified"}
}
},
"PatchObjectOutput":{
"type":"structure",
"members":{
"Object":{"shape":"PatchedObjectInfo"}
}
},
"PatchObjectRequest":{
"type":"structure",
"required":[
"Bucket",
"Key",
"ContentRange"
],
"members":{
"Body":{
"shape":"Body",
"streaming":true
},
"Bucket":{
"shape":"BucketName",
"location":"uri",
"locationName":"Bucket"
},
"ContentLength":{
"shape":"ContentLength",
"location":"header",
"locationName":"Content-Length"
},
"ContentMD5":{
"shape":"ContentMD5",
"location":"header",
"locationName":"Content-MD5"
},
"ContentRange":{
"shape":"ContentRange",
"location":"header",
"locationName":"Content-Range"
},
"IfMatch":{
"shape":"IfMatch",
"location":"header",
"locationName":"If-Match"
},
"IfUnmodifiedSince":{
"shape":"IfUnmodifiedSince",
"location":"header",
"locationName":"If-Unmodified-Since"
},
"Key":{
"shape":"ObjectKey",
"location":"uri",
"locationName":"Key"
},
"PatchAppendPartSize":{
"shape":"PatchAppendPartSize",
"location":"header",
"locationName":"X-Yc-S3-Patch-Append-Part-Size"
}
},
"payload":"Body"
},