Sunday, 8 September 2013

Efficiently checking if arbitrary object is NaN in Python / numpy / pandas?

Efficiently checking if arbitrary object is NaN in Python / numpy / pandas?

My numpy arrays use np.nan to designate missing values. As I iterate over
the data set, I need to detect such missing values and handle them in
special ways.
Naively I used numpy.isnan(val), which works well unless val isn't among
the subset of types supported by numpy.isnan(). For example, missing data
can occur in string fields, in which case I get:
>>> np.isnan('some_string')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Not implemented for this type
Other than writing an expensive wrapper that catches the exception and
returns False, is there a way to handle this elegantly and efficiently?

No comments:

Post a Comment