pyspark.sql.functions.nanvl

pyspark.sql.functions.nanvl(col1: ColumnOrName, col2: ColumnOrName) → pyspark.sql.column.Column

Returns col1 if it is not NaN, or col2 if col1 is NaN.

Both inputs should be floating point columns (DoubleType or FloatType).

New in version 1.6.0.

Examples

>>> df = spark.createDataFrame([(1.0, float('nan')), (float('nan'), 2.0)], ("a", "b"))
>>> df.select(nanvl("a", "b").alias("r1"), nanvl(df.a, df.b).alias("r2")).collect()
[Row(r1=1.0, r2=1.0), Row(r1=2.0, r2=2.0)]

pyspark.sql.functions.monotonically_increasing_id pyspark.sql.functions.rand