CIRCL
/
PyCIRCLean
mirror of https://github.com/CIRCL/PyCIRCLean
2
0
Fork 0
Code Issues Releases Wiki Activity
288 Commits
6 Branches
5 Tags
3.9 MiB
Commit Graph

80 Commits (538f7acc049666a6e3cc13d17b5a6bbb297179fc)

Author SHA1 Message Date
Dan Puttick 92d77e1a82 Revert change to magic.from_file 2017-07-20 18:40:16 -04:00
Dan Puttick 4bc113cefa Open file and magic.read_buffer in helpers.py 2017-07-20 18:40:16 -04:00
Dan Puttick 3eff10fbcd Unicode error check travis 2017-07-20 18:40:16 -04:00
Dan Puttick dee35a5331 Fix description string behavior 
* Make description string property return a string instead of array
 2017-07-20 18:40:16 -04:00
Dan Puttick 35eb8ea8ab Fixups for PR #16 
* ObjectPool winoffice files are now make_dangerous
* safe_copy now catches IOErrors only
* Use os.makedirs(exist_ok=True) instead of checking for existence in safe_copy
and create_metadata_file
* Added stubs for two tests related to safe_copy
 2017-07-17 14:52:22 -04:00
Dan Puttick 5aaeac6f16 Refactor FileBase.extension 2017-07-17 10:10:21 -04:00
Dan Puttick a47a00d7a9 Fix bug in split_mimetype and add test 2017-07-16 14:25:16 -04:00
Dan Puttick d7280a8535 Add tests for make_dangerous and safe_copy 2017-07-14 17:37:30 -04:00
Dan Puttick b77451ae7a Description_string now only allows strings 2017-07-14 16:42:31 -04:00
Dan Puttick df97228a75 Fix symlink behavior in helpers.py 2017-07-13 17:36:43 -04:00
Dan Puttick 27cbdc57bc Fix FileBase.make_dangerous filename editing 
* make_dangerous should only modify the filename if the file
wasn't previously identified as dangerous
* add_description should only be called if a reason_string
is supplied
 2017-07-13 12:55:26 -04:00
Dan Puttick e19064c83f Refactor FileBase to store props as attributes 
* This is kind of a big refactoring - I realized that storing file
props in a dict was causing some subtle problems, and that just having
them as attributes makes things a lot more simple
* I considered making a separate FileProps object and nesting it
inside FileBase, but almost all FileBase methods concern manipulating
file props, so it didn't really make sense.
* Tests are almost passing with this commit, but need a few more changes
and fixes for full test coverage + all passing.
 2017-07-12 17:58:39 -04:00
Dan Puttick f424be79cd Move KittenGroomerError to end of file 2017-07-11 15:48:40 -04:00
Dan Puttick 7b8b175a93 Fixup comments, add stubs for new tests 2017-07-11 15:48:40 -04:00
Dan Puttick 141a64cf33 Remove extra comments from helpers.py 
* Was thinking about removing processdir and main, but somebody might still
want to use them + they could be a helpful reference for a new person.
 2017-06-27 12:13:29 -04:00
Adam Schettenhelm 7b990df436 Check against value for duplicates when inserting into description_string 2017-05-26 20:37:36 -04:00
Dan Puttick 5865ddd94d Make root paths abspaths 
* src_root_path and dst_root_path are now converted to abspaths when
initializing kittengroomer to avoid any weird issues with relative or misformed
paths
 2017-04-10 13:22:36 +02:00
Dan Puttick c43ac0697a Add comments/notes to helpers.py 2017-04-10 13:22:36 +02:00
Dan Puttick 3f49612a23 Add new logger, move logging to filecheck 
* Wrote a new text-based logger that displays all file information in the tree
instead of using two separate logs
* Stopped using twiggy since it wasn't giving us anything useful
* Moved a lot of the logging code to filecheck, since it didn't really seem
appropriate as an API. Left a Logging stub in kittengroomer to hold methods
that might be useful for implementing other loggers.
* For the new logger, had to change the way that we traverse the items in the
source file tree.
 2017-04-10 13:22:20 +02:00
Dan Puttick ba407219d3 Open log file in text mode instead of bytes mode 2017-03-22 21:49:48 -04:00
Dan Puttick 52bb566cc3 Write basic log to log file 2017-03-22 12:07:43 -04:00
Dan Puttick 265f32ad6b Change the way description strings are handled 2017-03-22 10:28:00 -04:00
Dan Puttick 3e7b38c5d4 Improve doc strings on FileBase 2017-03-21 18:58:17 -04:00
Dan Puttick 6851461755 Change from two separate logs to one 2017-03-20 16:10:57 -04:00
Dan Puttick 51760ebbb1 Move default log setup back into filecheck 
* Realized that the API consumer might want to write their own logging tool.
* FileBase and KittenGroomerBase will have no logging code.
* If the API consumer likes, they can import GroomerLogger and use it in their
implementation.
 2017-03-20 16:10:57 -04:00
Dan Puttick 18857c8cf7 Change names of KittenGroomerBase root dir paths 2017-03-20 16:10:57 -04:00
Dan Puttick 27f58a0ede Put log dir creation code in separate method 2017-03-20 16:10:57 -04:00
Dan Puttick 4d8a1d1daf Add/update docstrings for filecheck and helpers 2017-03-15 22:56:00 -04:00
Dan Puttick 0175ee48e5 Add TODOs and clarify various logging messages 2017-03-15 22:56:00 -04:00
Dan Puttick 963a2feef4 Change various methods to properties 2017-03-15 22:55:51 -04:00
Dan Puttick 59cde8cfd5 Move safe_copy to FileBase 2017-03-15 21:06:07 -04:00
Dan Puttick e73721e95f Fix bug with safe_copy 2017-03-15 21:06:07 -04:00
Dan Puttick 484c71fc86 Turn off copying for certain mimes in filecheck 2017-03-15 21:06:07 -04:00
Dan Puttick 18857da7ca Several small bugfixes 
* Fix issue with main/subtypes in init
* Fix bug in File.check() in filecheck.py
* Fix FileBase.size for symlinks
 2017-03-15 21:06:07 -04:00
Dan Puttick 3fe8c7c223 Adjust order of property initialization 
Tests were failing due to values being set before file_props dict
was created
 2017-03-15 21:06:07 -04:00
Dan Puttick 0038d3ef66 Switch to using file properties 
* make_dangerous now takes a description string
* add_file_string takes strings describing the file
 2017-03-15 21:06:07 -04:00
Dan Puttick fc8923fddd Change Groomer private methods to public 
* Changed safe_rmtree, safe_copy, safe_remove, and safe_mkdir to public methods
* If something is being used in a subclass it probably shouldn't be a private
method
 2017-03-15 21:06:07 -04:00
Dan Puttick 12d5624b4d Change FileBase.log_details to Filebase._file_props 
* _file_props is a dict that will hold all information about the file
* Updated filecheck.py to reflect this
* Potentially will change contents of file_props to being attributes on the
file in the future. This change would be easy since all access to _file_props
is now via set_property and get_property methods.
* Add filename to _file_props
 2017-03-15 21:06:06 -04:00
Dan Puttick 1c58a7347e If no extentions FileBase.ext is now None 2017-03-15 21:06:06 -04:00
Dan Puttick b6c01db1fb Split mimetype methods 
- Instead of one large function that mutates FileBase properties,
mimetype and main/subtype are determined by two separate methods
that return mimetypes.
- The API is not changed.
- Absence of mimetype is now None instead of an empty string.
 2017-03-15 21:06:06 -04:00
Dan Puttick 9832101c85 Identify TODOs that are log related 2017-03-15 21:06:06 -04:00
Dan Puttick 781d0a76af First working version with methods in File object 
- All tests now passing with file handling methods on File object
instead of Groomer object.
- Logging functionality still isn't finished.
 2017-03-15 21:06:06 -04:00
Dan Puttick 9aafe6e518 Remove cur_file from methods in File object 2017-03-15 21:06:06 -04:00
Dan Puttick 3d36c90d66 Make list_all_files a public method 2017-03-15 21:04:57 -04:00
Dan Puttick 4a99ba900a Make self.get_extension return a value 2017-03-15 21:04:57 -04:00
Dan Puttick c6ecc5e3a3 Fix process_dir bug in filecheck tests 2017-03-15 21:04:57 -04:00
Dan Puttick bfa257534a Remove old data/ path stuff from helpers.py 2017-03-15 21:04:57 -04:00
Dan Puttick 61aa14c98d Change _write_log to _print_log 2017-03-15 21:04:57 -04:00
Dan Puttick 9d2f5c7241 Add .jpg test file, testfile_directory.md 
- Also fixed a bug with metadata code
 2017-03-15 21:04:57 -04:00
Dan Puttick 7d62238270 Hacks to make tests pass before fixing 2017-03-15 21:04:57 -04:00