CIRCL
/
PyCIRCLean
mirror of https://github.com/CIRCL/PyCIRCLean
2
0
Fork 0
Code Issues Releases Wiki Activity
214 Commits
6 Branches
5 Tags
3.9 MiB
Commit Graph

64 Commits (053f30db932d9b4a9a8956fb4a4ebe29cfd14a45)

Author SHA1 Message Date
Dan Puttick 5865ddd94d Make root paths abspaths 
* src_root_path and dst_root_path are now converted to abspaths when
initializing kittengroomer to avoid any weird issues with relative or misformed
paths
 2017-04-10 13:22:36 +02:00
Dan Puttick c43ac0697a Add comments/notes to helpers.py 2017-04-10 13:22:36 +02:00
Dan Puttick 3f49612a23 Add new logger, move logging to filecheck 
* Wrote a new text-based logger that displays all file information in the tree
instead of using two separate logs
* Stopped using twiggy since it wasn't giving us anything useful
* Moved a lot of the logging code to filecheck, since it didn't really seem
appropriate as an API. Left a Logging stub in kittengroomer to hold methods
that might be useful for implementing other loggers.
* For the new logger, had to change the way that we traverse the items in the
source file tree.
 2017-04-10 13:22:20 +02:00
Dan Puttick ba407219d3 Open log file in text mode instead of bytes mode 2017-03-22 21:49:48 -04:00
Dan Puttick 52bb566cc3 Write basic log to log file 2017-03-22 12:07:43 -04:00
Dan Puttick 265f32ad6b Change the way description strings are handled 2017-03-22 10:28:00 -04:00
Dan Puttick 3e7b38c5d4 Improve doc strings on FileBase 2017-03-21 18:58:17 -04:00
Dan Puttick 6851461755 Change from two separate logs to one 2017-03-20 16:10:57 -04:00
Dan Puttick 51760ebbb1 Move default log setup back into filecheck 
* Realized that the API consumer might want to write their own logging tool.
* FileBase and KittenGroomerBase will have no logging code.
* If the API consumer likes, they can import GroomerLogger and use it in their
implementation.
 2017-03-20 16:10:57 -04:00
Dan Puttick 18857c8cf7 Change names of KittenGroomerBase root dir paths 2017-03-20 16:10:57 -04:00
Dan Puttick 27f58a0ede Put log dir creation code in separate method 2017-03-20 16:10:57 -04:00
Dan Puttick 4d8a1d1daf Add/update docstrings for filecheck and helpers 2017-03-15 22:56:00 -04:00
Dan Puttick 0175ee48e5 Add TODOs and clarify various logging messages 2017-03-15 22:56:00 -04:00
Dan Puttick 963a2feef4 Change various methods to properties 2017-03-15 22:55:51 -04:00
Dan Puttick 59cde8cfd5 Move safe_copy to FileBase 2017-03-15 21:06:07 -04:00
Dan Puttick e73721e95f Fix bug with safe_copy 2017-03-15 21:06:07 -04:00
Dan Puttick 484c71fc86 Turn off copying for certain mimes in filecheck 2017-03-15 21:06:07 -04:00
Dan Puttick 18857da7ca Several small bugfixes 
* Fix issue with main/subtypes in init
* Fix bug in File.check() in filecheck.py
* Fix FileBase.size for symlinks
 2017-03-15 21:06:07 -04:00
Dan Puttick 3fe8c7c223 Adjust order of property initialization 
Tests were failing due to values being set before file_props dict
was created
 2017-03-15 21:06:07 -04:00
Dan Puttick 0038d3ef66 Switch to using file properties 
* make_dangerous now takes a description string
* add_file_string takes strings describing the file
 2017-03-15 21:06:07 -04:00
Dan Puttick fc8923fddd Change Groomer private methods to public 
* Changed safe_rmtree, safe_copy, safe_remove, and safe_mkdir to public methods
* If something is being used in a subclass it probably shouldn't be a private
method
 2017-03-15 21:06:07 -04:00
Dan Puttick 12d5624b4d Change FileBase.log_details to Filebase._file_props 
* _file_props is a dict that will hold all information about the file
* Updated filecheck.py to reflect this
* Potentially will change contents of file_props to being attributes on the
file in the future. This change would be easy since all access to _file_props
is now via set_property and get_property methods.
* Add filename to _file_props
 2017-03-15 21:06:06 -04:00
Dan Puttick 1c58a7347e If no extentions FileBase.ext is now None 2017-03-15 21:06:06 -04:00
Dan Puttick b6c01db1fb Split mimetype methods 
- Instead of one large function that mutates FileBase properties,
mimetype and main/subtype are determined by two separate methods
that return mimetypes.
- The API is not changed.
- Absence of mimetype is now None instead of an empty string.
 2017-03-15 21:06:06 -04:00
Dan Puttick 9832101c85 Identify TODOs that are log related 2017-03-15 21:06:06 -04:00
Dan Puttick 781d0a76af First working version with methods in File object 
- All tests now passing with file handling methods on File object
instead of Groomer object.
- Logging functionality still isn't finished.
 2017-03-15 21:06:06 -04:00
Dan Puttick 9aafe6e518 Remove cur_file from methods in File object 2017-03-15 21:06:06 -04:00
Dan Puttick 3d36c90d66 Make list_all_files a public method 2017-03-15 21:04:57 -04:00
Dan Puttick 4a99ba900a Make self.get_extension return a value 2017-03-15 21:04:57 -04:00
Dan Puttick c6ecc5e3a3 Fix process_dir bug in filecheck tests 2017-03-15 21:04:57 -04:00
Dan Puttick bfa257534a Remove old data/ path stuff from helpers.py 2017-03-15 21:04:57 -04:00
Dan Puttick 61aa14c98d Change _write_log to _print_log 2017-03-15 21:04:57 -04:00
Dan Puttick 9d2f5c7241 Add .jpg test file, testfile_directory.md 
- Also fixed a bug with metadata code
 2017-03-15 21:04:57 -04:00
Dan Puttick 7d62238270 Hacks to make tests pass before fixing 2017-03-15 21:04:57 -04:00
Dan Puttick 1cf8a62f46 First commit with Logger object 
- Made logger object
- Moved some logger related code from Groomer to Logger
- Changed logging related tests
- Filecheck tests still do not pass
 2017-03-15 21:04:57 -04:00
Dan Puttick 92d1b1cd93 Refactor metadata processing code 2017-03-15 21:01:28 -04:00
Dan Puttick e2af701ac9 Remove several pieces of unused code 
* Remove python 2 KittenGroomerBase.tree
* Remove default source and dest from KittenGroomerFileCheck
* Remove unused sys import
 2017-03-15 21:01:28 -04:00
Dan Puttick 774095d95a Fix list_all_files docstring 2017-03-15 21:01:28 -04:00
Dan Puttick 0be926c0eb Reorganize KittenGroomerBase __init__ 2017-03-15 21:01:28 -04:00
Raphaël Vinot 515687b1fe Generate SHA256 hash instead of SHA1. 2017-03-10 12:20:57 +01:00
Dan Puttick d06832e524 Remove kittengroomer/data/ 2017-01-19 16:11:58 -05:00
Dan Puttick f7ab393eb6 Refactor FileBase helper methods 2017-01-18 15:51:54 -05:00
Dan Puttick d4bfe794be In FileBase, self.extension is now always lowercase 2017-01-18 15:51:54 -05:00
Dan Puttick 09973488eb Added small comment to helpers.py 2016-12-16 16:10:59 -05:00
Dan Puttick 0364e038e1 Added unit tests for KittenGroomerBase 2016-12-14 15:44:59 -05:00
Dan Puttick 27d15254f4 Added/changed docstrings for KittenGroomerBase 2016-12-12 17:03:38 -05:00
Dan Puttick 62a7f680b4 Added/changed docstrings to FileBase 2016-12-09 11:28:38 -05:00
Dan Puttick 146f258e54 Refactored mimetype code into its own method 2016-12-09 11:28:38 -05:00
Raphaël Vinot abb565094c Fix py3 support 2016-05-10 17:16:04 +02:00
Raphaël Vinot 3efa57f0e2 write logfile as binary, use system packages when possible 2016-05-09 18:56:30 +02:00