May 13, 2023

Pyspark coalesce()

 

coalesce()

  • coalesce('col') -> returns first non-null value

  • If null, return the default value set 

    • lit(0) in this case


calculate payment bonus using coalesce()

  • payment = salary + (salary * bonus/100)

  • No bonus, default it to 0

  • coalesce ( col('bonus').cast('int'), lit(0) )  / 100



No comments:

Post a Comment