Skip to content
GitLab
Projects
Groups
Snippets
Help
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Open sidebar
brivas
mdf_reader
Commits
63b68cdc
Commit
63b68cdc
authored
5 years ago
by
iregon
Browse files
Options
Download
Email Patches
Plain Diff
Cleaned
parent
97a1ae67
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
5 deletions
+5
-5
reader/import_data.py
reader/import_data.py
+5
-5
No files found.
reader/import_data.py
View file @
63b68cdc
...
...
@@ -10,7 +10,7 @@ INPUT IS EITHER:
- pd.io.parsers.textfilereader
- io.StringIO
- file path
OUTPUT IS AN ITERABLE, DEPENDING ON SOURCE TYPE AND CHUNKSIZE BEING SET:
- a single dataframe in a list
- a pd.io.parsers.textfilereader
...
...
@@ -18,7 +18,7 @@ OUTPUT IS AN ITERABLE, DEPENDING ON SOURCE TYPE AND CHUNKSIZE BEING SET:
WITH BASICALLY 1 RECORD (ONE OR MULTIPLE REPORTS) IN ONE LINE
delimiter="
\t
" option in pandas.read_fwf avoids white spaces at tail
d
delimiter="
\t
" option in pandas.read_fwf avoids white spaces at tail
s
to be stripped
@author: iregon
...
...
@@ -26,7 +26,7 @@ to be stripped
DEV NOTES:
1) What this module is able to ingest needs to align with properties.supported_sources
2) Check io.StringIO input: why there, does it actually work as it is?
3) Check pd.io.parsers.textfilereader input: why there, does it actually work as it is?
3) Check pd.io.parsers.textfilereader input: why there, does it actually work as it is?
OPTIONS IN OLD DEVELOPMENT:
...
...
@@ -53,8 +53,8 @@ def to_iterable_df(source,skiprows = None, chunksize = None):
TextParser
=
pd
.
read_fwf
(
source
,
widths
=
[
properties
.
MAX_FULL_REPORT_WIDTH
],
header
=
None
,
delimiter
=
"
\t
"
,
skiprows
=
skiprows
,
chunksize
=
chunksize
)
if
not
chunksize
:
TextParser
=
[
TextParser
]
return
TextParser
return
TextParser
def
import_data
(
source
,
chunksize
=
None
,
skiprows
=
None
):
...
...
This diff is collapsed.
Click to expand it.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment