Skip to content
GitLab
Projects
Groups
Snippets
Help
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Open sidebar
brivas
mdf_reader
Commits
96591324
Commit
96591324
authored
5 years ago
by
iregon
Browse files
Options
Download
Email Patches
Plain Diff
Change output dtypes to object when datetime
parent
f945403d
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
1 deletion
+2
-1
read.py
read.py
+2
-1
No files found.
read.py
View file @
96591324
...
...
@@ -73,11 +73,12 @@ def ERV(TextParser,read_sections_list, schema, code_tables_path):
# This way it supports direct chunksize property inheritance if the input source was a pd.io.parsers.TextFileReader
chunksize
=
TextParser
.
orig_options
[
'chunksize'
]
if
isinstance
(
TextParser
,
pd
.
io
.
parsers
.
TextFileReader
)
else
None
# 'datetime' is not a valid pandas dtype: Only on output (on reading) will be then converted (via parse_dates) to datetime64[ns] type,
# cannot specify 'datetime' (of any kind) here: would fail
# cannot specify 'datetime' (of any kind) here: would fail
, need to change to 'object' and tell the date parser where it is
date_columns
=
[]
# Needs to be the numeric index of the column, as seems not to be able to work with tupples....
for
i
,
element
in
enumerate
(
list
(
out_dtypes
)):
if
out_dtypes
.
get
(
element
)
==
'datetime'
:
date_columns
.
append
(
i
)
out_dtypes
.
update
({
element
:
'object'
})
data
=
pd
.
read_csv
(
data_buffer
,
names
=
data_df
.
columns
,
chunksize
=
chunksize
,
dtype
=
out_dtypes
,
parse_dates
=
date_columns
)
valid
=
pd
.
read_csv
(
valid_buffer
,
names
=
data_df
.
columns
,
chunksize
=
chunksize
)
...
...
This diff is collapsed.
Click to expand it.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment