Update category URL filtering issue #147 #148
Conversation
PaloAltoNetworks#147 Proposed changes to update field category extraction Changes applied transforms.conf file: -extended capturing for report extract_threat to include new field extraction new_category props.conf file: -re-evaluate category from new_category and threat_category fields
|
Pretty keen on seeing this item get pushed through ASAP. |
|
+1 for moving this as quickly as possible please PAN team. |
|
+1 here also |
|
Great job @linsmeyerh ! |
|
+1 please push this through |
|
looking forward to seeing this implemented soon |
|
+1 to implement this |
| @@ -106,7 +106,7 @@ EVAL-report_id = if(log_subtype=="wildfire", coalesce(report_id,threat_id) | |||
| EVAL-http_category = if(log_subtype=="url", raw_category, null()) | |||
| EVAL-verdict = if(log_subtype=="wildfire", raw_category, null()) | |||
| EVAL-threat_category = if(log_subtype!="url" AND log_subtype!="file", if(threat_category=="unknown",log_subtype,coalesce(threat_category,log_subtype)), null()) | |||
| EVAL-category = if(log_subtype=="url" OR log_subtype=="file", raw_category, threat_category) | |||
| EVAL-category = if(log_subtype=="url" OR log_subtype=="file", split(new_category, ","), threat_category) | |||
There was a problem hiding this comment.
This doesn't seem compatible with older PAN-OS versions that don't have a new_category field. Perhaps you can coalesce raw_category and new_category first?
There was a problem hiding this comment.
Example:
EVAL-category = if(log_subtype=="url" OR log_subtype=="file", split(coalesce(new_category, raw_category), ","), threat_category)
There was a problem hiding this comment.
Hi @btorresgil ,
Thank you for looking after this issue/enhancement. The adjustment using coalesce making this great change compatible with older PAN-OS versions!!.
Thank you
There was a problem hiding this comment.
Hi @linsmeyerh just following up on this PR. If you could make the requested changes and push them to your remote branch. I can move this PR along.
| @@ -74,7 +74,7 @@ FORMAT = sourcetype::pan:config_traps | |||
|
|
|||
| [extract_threat] | |||
| DELIMS = "," | |||
| FIELDS = "future_use1","receive_time","serial_number","type","log_subtype","version","generated_time","src_ip","dest_ip","src_translated_ip","dest_translated_ip","rule","src_user","dest_user","app","vsys","src_zone","dest_zone","src_interface","dest_interface","log_forwarding_profile","future_use3","session_id","repeat_count","src_port","dest_port","src_translated_port","dest_translated_port","session_flags","transport","action","misc","threat","raw_category","severity","direction","sequence_number","action_flags","src_location","dest_location","future_use4","content_type","pcap_id","file_hash","cloud_address","url_index","user_agent","file_type","xff","referrer","sender","subject","recipient","report_id","devicegroup_level1","devicegroup_level2","devicegroup_level3","devicegroup_level4","vsys_name","dvc_name","future_use5","src_vm","dest_vm","http_method","tunnel_id","tunnel_monitor_tag","tunnel_session_id","tunnel_start_time","tunnel_type","threat_category","content_version" | |||
| FIELDS = "future_use1","receive_time","serial_number","type","log_subtype","version","generated_time","src_ip","dest_ip","src_translated_ip","dest_translated_ip","rule","src_user","dest_user","app","vsys","src_zone","dest_zone","src_interface","dest_interface","log_forwarding_profile","future_use3","session_id","repeat_count","src_port","dest_port","src_translated_port","dest_translated_port","session_flags","transport","action","misc","threat","raw_category","severity","direction","sequence_number","action_flags","src_location","dest_location","future_use4","content_type","pcap_id","file_hash","cloud_address","url_index","user_agent","file_type","xff","referrer","sender","subject","recipient","report_id","devicegroup_level1","devicegroup_level2","devicegroup_level3","devicegroup_level4","vsys_name","dvc_name","future_use5","src_vm","dest_vm","http_method","tunnel_id","tunnel_monitor_tag","tunnel_session_id","tunnel_start_time","tunnel_type","threat_category","content_version","future_use5","future_use6","future_use7","future_use8","new_category" | |||
There was a problem hiding this comment.
The name new_category is relative because newness is lost over time. Can you use a descriptive name for the field such as url_category?
There was a problem hiding this comment.
I agree 100% with new field name url_category and think the best way to go.
Thank you.
|
Note to maintainers: No changes to the datamodel necessary because the new field is coalesced to the |
|
@linsmeyerh If could you resolve the requested changes by updating your code. I could put this into the next release. |
|
hope this isn't too late +1 to push this |
|
@linsmeyerh If you can make the requested changes we can merge this in and release it. If not, we can close this PR and make the changes ourselves. Let us know. |
|
@btorresgil thank you. I'm happy for you to close this PR and make the changes. Please include this work in the next release!! |
|
Closing this pull request as changes are part of PR #154 |
Hi Team,
This PR is for the issue raised #147
Proposed changes to update field
category- extracting frompan:threatraw logs.Changes applied
transforms.conf file:
-extend capturing for report extract_threat to include new field extraction
new_categoryprops.conf file:
-re-evaluate category from new_category and threat_category fields