GitHub

[Python] Column with over 2GB size limit but still identified as String in schema

I read online that in PyArrow a string column would have a column-level size limit of 2GB. However, in my work I noticed this doesn't hold. def some_function( self, raw_table: pa.Table, ): schema = ...