Skip to main content

Mapping S3 object metadata to Pics.io asset fields

Learn how to automatically populate your asset's Title and custom fields from S3 object metadata during sync — no manual re-entry needed.

Written by Denis Limishko

If you store useful information as user-defined S3 object metadata (such as x-amz-meta-* headers your upstream system writes onto each file), Pics.io can copy that metadata into the matching asset's Title and custom fields during sync.

That way, your existing pipeline stays the source of truth and your team gets a searchable, filterable view inside Pics.io without manually re-entering anything.

NB! This feature is enabled per account by our support team. To turn it on, send your completed mapping file to support@pics.io.

Once the feature is enabled and a mapping is in place:

  1. Pics.io syncs your S3 bucket as usual.

  2. For every new asset that sync creates, Pics.io reads the object's S3 metadata.

  3. For each metadata key listed in your mapping, the value is written into the corresponding Pics.io field — either the asset's Title or one of your custom fields.

  4. The rest of the sync (thumbnails, keywording, permissions) continues normally.

After the first sync, you can filter, search, and group assets on those custom fields just like any other metadata in Pics.io.

Requirements

  • Your library uses Amazon S3 (or an S3-compatible bucket) as its storage backend.

  • The objects in the bucket already carry the metadata you want to import — set by your uploader, ETL pipeline, or another upstream system.

  • You have custom fields already configured in Pics.io for every destination you want to populate. The mapping can only fill custom fields that exist in your schema; unknown destinations are silently skipped at sync time.

How the mapping works

Each row in the mapping pairs one S3 metadata key (the Source) with one Pics.io custom field (the Destination) that you've set up. Some examples:

Source: S3 metadata key

Destination

What happens

display_name

Title

The value is copied into the asset's Title field.

creator

Creator_custom (a custom field you created)

The value is copied into the Creator_custom custom field.

manufacturing_city

City_custom (a custom field you created)

The value is copied into the City_custom custom field.

A few rules to keep in mind when preparing your mapping:

  • Source keys must be lowercase. S3 normalizes all user-defined metadata keys to lowercase when it returns an object, so write your S3 keys in lowercase to match what Pics.io actually receives.

  • Destinations are matched against the custom field's display name (its title in Pics.io), case-insensitively. If you have a custom field called "Creator Full Name", you can write it as Creator Full Name, creator full name, or CREATOR FULL NAME — all will match.

  • Only Title and custom fields are supported as destinations. You cannot map S3 metadata into system-managed fields such as upload date, file size, or storage path.

  • Non-ASCII values are supported. If your S3 metadata uses RFC 2047 encoding (e.g. =?UTF-8?B?Sm9obiBEb2U=?=) to carry accented characters or Cyrillic text, Pics.io decodes them automatically before writing to the field.

What gets mapped, and when

Scenario

Behavior

New asset created by sync, with S3 metadata that matches your mapping

The destination Title / custom field is populated.

New asset created by sync, with S3 metadata not in your mapping

Ignored.

New asset created by sync, but no S3 metadata is set on the object

Asset is created normally; no metadata fields are filled.

Asset that was already synced before this feature was enabled

Not touched. The mapping only applies to assets that sync creates after the feature is enabled.

Mapping references a custom field that doesn't exist in your schema

That single entry is silently skipped; all other mapping entries still apply.

Same S3 metadata key listed twice with different destinations

Not allowed — support will reject a mapping that contains duplicate source keys.

NB! Backfilling existing assets: If you've already synced your bucket and want to apply the mapping to assets that already exist in Pics.io, contact support — that requires a separate one-off job.

How to enable it for your account

  1. Make sure every custom field you plan to populate already exists in your Pics.io schema. Go to Settings → Custom Fields and create any missing fields. (How to manage custom fields.)

  2. Create the mapping template (CSV) and fill it in — see instructions in Filling the template below.

  3. Send the completed CSV to the support team, along with the email address of the account owner whose library should receive the mapping.

  4. Support team will validate the file, enable the feature on your account, and confirm when it's ready. From that point on, every sync will apply the mapping to newly-created assets.

To change the mapping later, send an updated CSV — it fully replaces the previous one.

Filling the template

The mapping template must be a CSV file with three columns:

  • Pics.io Custom Field

    • The exact title of the custom field as it appears in your Pics.io schema, or the literal word Title to populate the asset's Title.

  • S3 Metadata Field

    • The S3 metadata key as set on the object, lowercased. Do not include the x-amz-meta- prefix.

  • Notes

    • This column can be left empty, but you can enter any additional details for your own reference and better understanding for us — useful for grouping sections or recording where data originates.

You may leave a row blank to add a visual separator between groups; blank rows are skipped.

Example — a completed mapping template:

Pics.io Custom Field

S3 Metadata Field

Notes

Title

displayname

Maps to the asset Title

Creator

creator

Creator Full Name

creatorfullname

May be RFC 2047 encoded

Creation Date

creationdate

City

manufacturing_city

GPS Latitude

latitude

From EXIF

GPS Longitude

longitude

From EXIF

With this mapping in place, an S3 object carrying the metadata below would produce the following result in Pics.io:

S3 metadata on the object

Result in Pics.io

displayname = Sunset.jpg

Asset Title → Sunset.jpg

creator = jdoe

Creator custom field → jdoe

creatorfullname = =?UTF-8?B?Sm9obiBEb2U=?=

Creator Full Name custom field → John Doe (decoded automatically)

manufacturing_city = Paris

City custom field → Paris

Troubleshooting

Values aren't appearing on my assets.

  • Confirm the assets are newly synced since the feature was enabled. The mapping doesn't backfill existing assets.

  • Confirm the custom field name in your mapping exactly matches the title in your Pics.io schema (spelling matters; case doesn't).

  • Confirm the S3 object actually carries the metadata key you expect — metadata is set at upload time. You can verify with aws s3api head-object --bucket <bucket> --key <key>.

  • Confirm the S3 key in your mapping is lowercase.

Non-ASCII characters look garbled (=?UTF-8?B?...?=).

  • Pics.io only decodes values that conform to RFC 2047 encoding. If your uploader uses a different encoding, contact support for guidance.

I added a new custom field after the mapping was enabled and it's not getting populated.

  • No change to the mapping CSV is needed — adding the field to your schema is enough. It will start populating on the next sync, provided the field title matches what's in your mapping.

I want to remove a mapping entry.

  • Send an updated CSV with the row removed. The new mapping fully replaces the old one.


FAQ

Does this affect upload speed or sync time?

  • A small extra request per asset is made to read its S3 metadata. In practice this is negligible against the rest of the sync work. Concurrency is throttled by Pics.io to stay within S3 rate limits.

Can I map the same S3 value into two custom fields?

  • Yes — list the same S3 source key on two rows with different destinations. However, if you need to do this, mention it explicitly in your email when sending the mapping so support can confirm the configuration is correct.

Can I map system metadata (Content-Type, ETag, etc.)?

  • No — only user-defined object metadata (the keys you set at upload, exposed under the Metadata block on GetObject) is supported. System headers are not available to the mapping.

Is this feature available for Google Drive sync?

  • No — this is S3-specific. Google Drive has its own metadata model and is not covered by this mapping.


Need help setting this up? Contact support@pics.io and we'll walk you through it.

Did this answer your question?