CIRCL
/
PyCIRCLean
mirror of https://github.com/CIRCL/PyCIRCLean
2
0
Fork 0
Code Issues Releases Wiki Activity
254 Commits
6 Branches
5 Tags
3.9 MiB
Commit Graph

70 Commits (df97228a7545ff84c3ece7ce9f2b927af5b1babd)

Author SHA1 Message Date
Dan Puttick df97228a75 Fix symlink behavior in helpers.py 2017-07-13 17:36:43 -04:00
Dan Puttick 27cbdc57bc Fix FileBase.make_dangerous filename editing 
* make_dangerous should only modify the filename if the file
wasn't previously identified as dangerous
* add_description should only be called if a reason_string
is supplied
 2017-07-13 12:55:26 -04:00
Dan Puttick e19064c83f Refactor FileBase to store props as attributes 
* This is kind of a big refactoring - I realized that storing file
props in a dict was causing some subtle problems, and that just having
them as attributes makes things a lot more simple
* I considered making a separate FileProps object and nesting it
inside FileBase, but almost all FileBase methods concern manipulating
file props, so it didn't really make sense.
* Tests are almost passing with this commit, but need a few more changes
and fixes for full test coverage + all passing.
 2017-07-12 17:58:39 -04:00
Dan Puttick f424be79cd Move KittenGroomerError to end of file 2017-07-11 15:48:40 -04:00
Dan Puttick 7b8b175a93 Fixup comments, add stubs for new tests 2017-07-11 15:48:40 -04:00
Dan Puttick 141a64cf33 Remove extra comments from helpers.py 
* Was thinking about removing processdir and main, but somebody might still
want to use them + they could be a helpful reference for a new person.
 2017-06-27 12:13:29 -04:00
Adam Schettenhelm 7b990df436 Check against value for duplicates when inserting into description_string 2017-05-26 20:37:36 -04:00
Dan Puttick 5865ddd94d Make root paths abspaths 
* src_root_path and dst_root_path are now converted to abspaths when
initializing kittengroomer to avoid any weird issues with relative or misformed
paths
 2017-04-10 13:22:36 +02:00
Dan Puttick c43ac0697a Add comments/notes to helpers.py 2017-04-10 13:22:36 +02:00
Dan Puttick 3f49612a23 Add new logger, move logging to filecheck 
* Wrote a new text-based logger that displays all file information in the tree
instead of using two separate logs
* Stopped using twiggy since it wasn't giving us anything useful
* Moved a lot of the logging code to filecheck, since it didn't really seem
appropriate as an API. Left a Logging stub in kittengroomer to hold methods
that might be useful for implementing other loggers.
* For the new logger, had to change the way that we traverse the items in the
source file tree.
 2017-04-10 13:22:20 +02:00
Dan Puttick ba407219d3 Open log file in text mode instead of bytes mode 2017-03-22 21:49:48 -04:00
Dan Puttick 52bb566cc3 Write basic log to log file 2017-03-22 12:07:43 -04:00
Dan Puttick 265f32ad6b Change the way description strings are handled 2017-03-22 10:28:00 -04:00
Dan Puttick 3e7b38c5d4 Improve doc strings on FileBase 2017-03-21 18:58:17 -04:00
Dan Puttick 6851461755 Change from two separate logs to one 2017-03-20 16:10:57 -04:00
Dan Puttick 51760ebbb1 Move default log setup back into filecheck 
* Realized that the API consumer might want to write their own logging tool.
* FileBase and KittenGroomerBase will have no logging code.
* If the API consumer likes, they can import GroomerLogger and use it in their
implementation.
 2017-03-20 16:10:57 -04:00
Dan Puttick 18857c8cf7 Change names of KittenGroomerBase root dir paths 2017-03-20 16:10:57 -04:00
Dan Puttick 27f58a0ede Put log dir creation code in separate method 2017-03-20 16:10:57 -04:00
Dan Puttick 4d8a1d1daf Add/update docstrings for filecheck and helpers 2017-03-15 22:56:00 -04:00
Dan Puttick 0175ee48e5 Add TODOs and clarify various logging messages 2017-03-15 22:56:00 -04:00
Dan Puttick 963a2feef4 Change various methods to properties 2017-03-15 22:55:51 -04:00
Dan Puttick 59cde8cfd5 Move safe_copy to FileBase 2017-03-15 21:06:07 -04:00
Dan Puttick e73721e95f Fix bug with safe_copy 2017-03-15 21:06:07 -04:00
Dan Puttick 484c71fc86 Turn off copying for certain mimes in filecheck 2017-03-15 21:06:07 -04:00
Dan Puttick 18857da7ca Several small bugfixes 
* Fix issue with main/subtypes in init
* Fix bug in File.check() in filecheck.py
* Fix FileBase.size for symlinks
 2017-03-15 21:06:07 -04:00
Dan Puttick 3fe8c7c223 Adjust order of property initialization 
Tests were failing due to values being set before file_props dict
was created
 2017-03-15 21:06:07 -04:00
Dan Puttick 0038d3ef66 Switch to using file properties 
* make_dangerous now takes a description string
* add_file_string takes strings describing the file
 2017-03-15 21:06:07 -04:00
Dan Puttick fc8923fddd Change Groomer private methods to public 
* Changed safe_rmtree, safe_copy, safe_remove, and safe_mkdir to public methods
* If something is being used in a subclass it probably shouldn't be a private
method
 2017-03-15 21:06:07 -04:00
Dan Puttick 12d5624b4d Change FileBase.log_details to Filebase._file_props 
* _file_props is a dict that will hold all information about the file
* Updated filecheck.py to reflect this
* Potentially will change contents of file_props to being attributes on the
file in the future. This change would be easy since all access to _file_props
is now via set_property and get_property methods.
* Add filename to _file_props
 2017-03-15 21:06:06 -04:00
Dan Puttick 1c58a7347e If no extentions FileBase.ext is now None 2017-03-15 21:06:06 -04:00
Dan Puttick b6c01db1fb Split mimetype methods 
- Instead of one large function that mutates FileBase properties,
mimetype and main/subtype are determined by two separate methods
that return mimetypes.
- The API is not changed.
- Absence of mimetype is now None instead of an empty string.
 2017-03-15 21:06:06 -04:00
Dan Puttick 9832101c85 Identify TODOs that are log related 2017-03-15 21:06:06 -04:00
Dan Puttick 781d0a76af First working version with methods in File object 
- All tests now passing with file handling methods on File object
instead of Groomer object.
- Logging functionality still isn't finished.
 2017-03-15 21:06:06 -04:00
Dan Puttick 9aafe6e518 Remove cur_file from methods in File object 2017-03-15 21:06:06 -04:00
Dan Puttick 3d36c90d66 Make list_all_files a public method 2017-03-15 21:04:57 -04:00
Dan Puttick 4a99ba900a Make self.get_extension return a value 2017-03-15 21:04:57 -04:00
Dan Puttick c6ecc5e3a3 Fix process_dir bug in filecheck tests 2017-03-15 21:04:57 -04:00
Dan Puttick bfa257534a Remove old data/ path stuff from helpers.py 2017-03-15 21:04:57 -04:00
Dan Puttick 61aa14c98d Change _write_log to _print_log 2017-03-15 21:04:57 -04:00
Dan Puttick 9d2f5c7241 Add .jpg test file, testfile_directory.md 
- Also fixed a bug with metadata code
 2017-03-15 21:04:57 -04:00
Dan Puttick 7d62238270 Hacks to make tests pass before fixing 2017-03-15 21:04:57 -04:00
Dan Puttick 1cf8a62f46 First commit with Logger object 
- Made logger object
- Moved some logger related code from Groomer to Logger
- Changed logging related tests
- Filecheck tests still do not pass
 2017-03-15 21:04:57 -04:00
Dan Puttick 92d1b1cd93 Refactor metadata processing code 2017-03-15 21:01:28 -04:00
Dan Puttick e2af701ac9 Remove several pieces of unused code 
* Remove python 2 KittenGroomerBase.tree
* Remove default source and dest from KittenGroomerFileCheck
* Remove unused sys import
 2017-03-15 21:01:28 -04:00
Dan Puttick 774095d95a Fix list_all_files docstring 2017-03-15 21:01:28 -04:00
Dan Puttick 0be926c0eb Reorganize KittenGroomerBase __init__ 2017-03-15 21:01:28 -04:00
Raphaël Vinot 515687b1fe Generate SHA256 hash instead of SHA1. 2017-03-10 12:20:57 +01:00
Dan Puttick f7ab393eb6 Refactor FileBase helper methods 2017-01-18 15:51:54 -05:00
Dan Puttick d4bfe794be In FileBase, self.extension is now always lowercase 2017-01-18 15:51:54 -05:00
Dan Puttick 09973488eb Added small comment to helpers.py 2016-12-16 16:10:59 -05:00