Skip to content

Commit 75db71c

Browse files
committed
FIX: read_csv: Re-convert to target dtype after filtering bad lines
1 parent 09d10d2 commit 75db71c

File tree

1 file changed

+10
-0
lines changed

1 file changed

+10
-0
lines changed

pandas/_libs/parsers.pyx

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1099,6 +1099,16 @@ cdef class TextReader:
10991099
stacklevel=find_stack_level()
11001100
)
11011101

1102+
# Re-convert failed columns to their target dtype now that bad rows
1103+
# have been removed. All remaining values should be convertible.
1104+
for col_idx, target_dtype in failed_columns_dtypes.items():
1105+
try:
1106+
results[col_idx] = np.array(results[col_idx]).astype(target_dtype)
1107+
except (ValueError, TypeError, OverflowError):
1108+
# If conversion still fails, keep as string (shouldn't happen
1109+
# if _identify_bad_rows worked correctly, but be defensive)
1110+
pass
1111+
11021112
self.parser_start += end - start
11031113

11041114
return results

0 commit comments

Comments
 (0)