Skip to content

Conversation

@kevinjqliu
Copy link
Contributor

@kevinjqliukevinjqliu commented Nov 9, 2025

Rationale for this change

Closes#2663
Relates to #2333 which allow reading pyarrow timestamp as iceberg timestamptz

This PR allows pyarrow timestamptz to be read as iceberg timestamp.

Although this configuration does not conform to the Iceberg spec, the change aligns PyIceberg's behavior with Spark when reading mismatched types (timestamptz as iceberg timestamp and timestamp as iceberg timestamptz)

Are these changes tested?

Yes

Are there any user-facing changes?

No

@Fokko
Copy link
Contributor

I'm comfortable with having this patch in the read path, but it looks like we're also implicitly adding this to the write path:

_to_requested_schema(

I'm reluctant to add it there since we want to enforce the difference between timestamp and timestamptz and not silently ignore it.

@kevinjqliu
Copy link
ContributorAuthor

thanks! restricting the write side makes sense. Let me take a look at how we can do that

@kevinjqliukevinjqliu deleted the kevinjqliu/read-parquet-timestamptz-as-iceberg-timestamp branch December 5, 2025 17:15
@kevinjqliukevinjqliu restored the kevinjqliu/read-parquet-timestamptz-as-iceberg-timestamp branch December 5, 2025 17:23
@kevinjqliukevinjqliu reopened this Dec 5, 2025
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

"Cannot promote timestamp to timestamptz" error when loading Dremio created table

2 participants

@kevinjqliu@Fokko