Skip to content

Commit 9dc8eea

Browse files
authored
Update readme.md
Cleaned up verbage and matched to this directories new purpose
1 parent dd49628 commit 9dc8eea

1 file changed

Lines changed: 3 additions & 6 deletions

File tree

‎readme.md‎

Lines changed: 3 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,12 @@
1-
# Basic-ETL
1+
# Basic-Backend
22

3-
This is an example final project of a local ETL process. As an intro & undergrad
4-
class this does not include network operations that would typically be expected and this version does not include NoSQL systems. It starts with building Tests (TDD) and then progresses through:
3+
Before students could be introduced to the actual ETL (Extract-Transfer-Load) process, it was nesscary to introduce backend engineering work. This contains both an overview of scripting techniques and some specific tools for processing *"standard"* files and an example project of standard serialization using json. As an intro & undergrad
4+
class this does not include network operations that would typically be expected nor NoSQL systems (these are covered in a seperate section). It starts with building Tests (TDD), full mock systems are found in a different course, and then progresses through:
55
- Checking a directory for files
66
- Processing the files in the directory based on their extension
77
- This example includes a parallel processing option (this was not required)
88
- Standard Parsing (remove empty lines and performs basic validation)
9-
- Text Files include very basic NLP operation (nGram & regex matching)
109
- Loads data to an included SQLite database
11-
- This was used to explain the limitations of ACID for analysis
12-
- Seperate version with MongoDB is included in its own Repository
1310
- Performs standard SQL analysis (using aggregated functions and group by)
1411
- Performs Rule Based Analysis (Affinity Analysis and Proability of event)
1512

0 commit comments

Comments
 (0)