You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Negative-scale decimal /, div, ceil and floor fall back to Spark while round dispatches #7023
CometRound returns Unsupported for the same input but mixes in CodegenDispatchFallback, so round stays in the Comet pipeline and runs Spark's own doGenCode (arithmetic.scala#L366-L382). round on negative-scale decimal in CometExpressionSuite covers it. The other four could do the same. The EPIC deferred ceil and floor as marginal, since only the legacy flag reaches them, and #5050 adds division to that group.
Fix: for CometCeil and CometFloor the negative scale is the only Unsupported case, so mixing in CodegenDispatchFallback is the whole change. For CometDivide and CometIntegralDivide it isn't a one-line change. The mixin would also send their other Unsupported results to the dispatcher, namely mathDataTypeSupportLevel's unsupported-datatype case and CometDivide's check that a decimal result has decimal operands. Each of those needs checking against the dispatcher's type gate, or these serdes need a way to dispatch only the negative-scale case.
Tests should cover a materialized negative-scale column, not only a cast literal. That input broke the dispatcher's decimal reader until #5050 made it read negative scales through BigDecimal.
Activity