Commit 63b68cdc authored by iregon's avatar iregon
Browse files

Cleaned

parent 97a1ae67
...@@ -10,7 +10,7 @@ INPUT IS EITHER: ...@@ -10,7 +10,7 @@ INPUT IS EITHER:
- pd.io.parsers.textfilereader - pd.io.parsers.textfilereader
- io.StringIO - io.StringIO
- file path - file path
OUTPUT IS AN ITERABLE, DEPENDING ON SOURCE TYPE AND CHUNKSIZE BEING SET: OUTPUT IS AN ITERABLE, DEPENDING ON SOURCE TYPE AND CHUNKSIZE BEING SET:
- a single dataframe in a list - a single dataframe in a list
- a pd.io.parsers.textfilereader - a pd.io.parsers.textfilereader
...@@ -18,7 +18,7 @@ OUTPUT IS AN ITERABLE, DEPENDING ON SOURCE TYPE AND CHUNKSIZE BEING SET: ...@@ -18,7 +18,7 @@ OUTPUT IS AN ITERABLE, DEPENDING ON SOURCE TYPE AND CHUNKSIZE BEING SET:
WITH BASICALLY 1 RECORD (ONE OR MULTIPLE REPORTS) IN ONE LINE WITH BASICALLY 1 RECORD (ONE OR MULTIPLE REPORTS) IN ONE LINE
delimiter="\t" option in pandas.read_fwf avoids white spaces at taild delimiter="\t" option in pandas.read_fwf avoids white spaces at tails
to be stripped to be stripped
@author: iregon @author: iregon
...@@ -26,7 +26,7 @@ to be stripped ...@@ -26,7 +26,7 @@ to be stripped
DEV NOTES: DEV NOTES:
1) What this module is able to ingest needs to align with properties.supported_sources 1) What this module is able to ingest needs to align with properties.supported_sources
2) Check io.StringIO input: why there, does it actually work as it is? 2) Check io.StringIO input: why there, does it actually work as it is?
3) Check pd.io.parsers.textfilereader input: why there, does it actually work as it is? 3) Check pd.io.parsers.textfilereader input: why there, does it actually work as it is?
OPTIONS IN OLD DEVELOPMENT: OPTIONS IN OLD DEVELOPMENT:
...@@ -53,8 +53,8 @@ def to_iterable_df(source,skiprows = None, chunksize = None): ...@@ -53,8 +53,8 @@ def to_iterable_df(source,skiprows = None, chunksize = None):
TextParser = pd.read_fwf(source,widths=[properties.MAX_FULL_REPORT_WIDTH],header = None, delimiter="\t", skiprows = skiprows, chunksize = chunksize) TextParser = pd.read_fwf(source,widths=[properties.MAX_FULL_REPORT_WIDTH],header = None, delimiter="\t", skiprows = skiprows, chunksize = chunksize)
if not chunksize: if not chunksize:
TextParser = [TextParser] TextParser = [TextParser]
return TextParser return TextParser
def import_data(source,chunksize = None, skiprows = None): def import_data(source,chunksize = None, skiprows = None):
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment