Repository navigation
Commit d6d425d
authored
Declare pyarrow for Parquet output of SqlToS3Operator (#74691)
Writing Parquet files and reading with the pyarrow dtype backend both need pyarrow, but the provider never declared it. It only worked when another installed package happened to bring pyarrow in, and otherwise failed at runtime with pandas' "Unable to find a usable engine" error. A separate extra keeps the roughly 40 MB dependency out of installs that only write CSV or JSON.1 parent a23aa81 commit d6d425d
4 files changed
Lines changed: 23 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
174 | 174 | | |
175 | 175 | | |
176 | 176 | | |
177 | | - | |
| 177 | + | |
178 | 178 | | |
179 | | - | |
| 179 | + | |
180 | 180 | | |
181 | 181 | | |
182 | 182 | | |
| |||
191 | 191 | | |
192 | 192 | | |
193 | 193 | | |
| 194 | + | |
194 | 195 | | |
195 | 196 | | |
196 | 197 | | |
| |||
199 | 200 | | |
200 | 201 | | |
201 | 202 | | |
202 | | - | |
| 203 | + | |
203 | 204 | | |
204 | 205 | | |
205 | 206 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
27 | 31 | | |
28 | 32 | | |
29 | 33 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
136 | 136 | | |
137 | 137 | | |
138 | 138 | | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
139 | 144 | | |
140 | 145 | | |
141 | 146 | | |
| |||
201 | 206 | | |
202 | 207 | | |
203 | 208 | | |
| 209 | + | |
204 | 210 | | |
205 | 211 | | |
206 | 212 | | |
| |||
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
0 commit comments