AniNIX/Kapisi
3
0
Fork 0
Code Issues 27 Pull Requests Projects 1 Releases Wiki Activity

Whitespace cleanup to get in sync with AniNIX/Uniglot hooks

Browse Source
This commit is contained in:
DarkFeather
2022-11-20 20:03:01 -06:00
parent a43cb4b6bb
commit a34c96df6b
96 changed files with 713 additions and 400 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
roles/Sharingan/README.md
View File

@@ -5,7 +5,7 @@ Sharingan is named after the mythical technique from the Naruto anime series. Sh
# Relevant Files and Software
We use Graylog on a dedicated VM to aggregate results. By default, all servers in a datacenter should send journald via syslog to `sharingan.$datacenter.aninix.net`.
We use Graylog on a dedicated VM to aggregate results. By default, all servers in a datacenter should send journald via syslog to `sharingan.$datacenter.aninix.net`.
## Syslog-ng
@@ -13,7 +13,7 @@ We use a lot of services in the AniNIX ecosystem -- some create files, some pipe
## Journald
ArchLinux and most systemd-based Linux distributions use journald to track system log files.
ArchLinux and most systemd-based Linux distributions use journald to track system log files.
## Suricata
@@ -32,12 +32,12 @@ TODO
## Graylog
## Elasticsearch
Elasticsearch acts as graylog's data backend.
Elasticsearch acts as graylog's data backend.
We have seen issues where poor disk i/o or unplanned shutdown can cause Elasticsearch to have index corruption.
We have seen issues where poor disk i/o or unplanned shutdown can cause Elasticsearch to have index corruption.
1. Stop elasticsearch
1. From `/usr/share/elasticsearch/lib`, you can use `java -cp lucene-core*.jar -ea:org.apache.lucene... org.apache.lucene.index.CheckIndex /usr/share/elasticsearch/data/nodes/0/indices/1nJc43t7TGuHmVR3Q5w9PA/1/index -verbose -exorcise` (on the right index) to exorcise the corrupted data.
1. From `/usr/share/elasticsearch/lib`, you can use `java -cp lucene-core*.jar -ea:org.apache.lucene... org.apache.lucene.index.CheckIndex /usr/share/elasticsearch/data/nodes/0/indices/1nJc43t7TGuHmVR3Q5w9PA/1/index -verbose -exorcise` (on the right index) to exorcise the corrupted data.
1. Remove corruption flags: `rm /usr/share/elasticsearch/data/nodes/0/indices/1nJc43t7TGuHmVR3Q5w9PA/1/index/corrupted_*`
1. Restart elasticsearch
1. Retry shard allocation:

2
roles/Sharingan/files/monit/checks/system
View File

@@ -7,7 +7,7 @@
## performed should a test fail.
check system $HOST
if loadavg (1min) per core > 2 for 25 times within 30 cycles then exec "/etc/monit.d/scripts/critical Load average is high"
if memory usage > 75% for 8 times within 10 cycles then exec "/etc/monit.d/scripts/critical Memory is overused."
if memory usage > 90% for 8 times within 10 cycles then exec "/etc/monit.d/scripts/critical Memory is overused."
if swap usage > 25% for 8 times within 10 cycles then exec "/etc/monit.d/scripts/critical Swap is overused."
group system
#

2
roles/Sharingan/files/monit/checks/warrant-canary Normal file
View File

@@ -0,0 +1,2 @@
check program warrant_canary with path "/etc/monit.d/scripts/check-warrant-canary"
if status != 0 then exec "/etc/monit.d/scripts/critical Warrant Canary needs to be updated"

1
roles/Sharingan/files/monit/hostdefs/Core
View File

@@ -1 +1,2 @@
include "/etc/monit.d/checks/system"
include "/etc/monit.d/checks/warrant-canary"

27
roles/Sharingan/files/monit/scripts/check-warrant-canary Executable file
View File

@@ -0,0 +1,27 @@
#!/bin/bash
daysinadvance=14
url=https://aninix.net/AniNIX/WarrantCanary/raw/branch/main/canary
# ID the update expected date
canarydate="$(curl -s "$url" | grep -A 1 'next two updates' | tail -n 1 | sed 's/^..//' )"
if [ -z "$canarydate" ]; then
echo "Cannot identify the canary's next time."
exit 2
fi
epochcanarydate="$(date -d "$canarydate" +%s)"
# Remove a one-month padding
updatecanarydate=$(( $epochcanarydate - 2592000 ))
# ID today
today="$(date +%s)"
if [[ $today -gt $updatecanarydate ]]; then
echo Time to reseed the warrant canary.
exit 2
else
echo Warrant canary seems to have the right time.
exit 0
fi

160
roles/Sharingan/files/oinkmaster/oinkmaster.conf
View File

@@ -1,11 +1,11 @@
# $Id: oinkmaster.conf,v 1.132 2006/02/02 12:05:08 andreas_o Exp $ #
# This file is pretty big by default, but don't worry.
# The only things required are "path" and "update_files". You must also
# set "url" to point to the correct rules archive for your version of
# This file is pretty big by default, but don't worry.
# The only things required are "path" and "update_files". You must also
# set "url" to point to the correct rules archive for your version of
# Snort, unless you prefer to specify this on the command line.
# The rest in here is just a few recommended defaults, and examples
# how to use all the other optional features and give some ideas how they
# how to use all the other optional features and give some ideas how they
# could be used.
# Remember not to let untrusted users edit Oinkmaster configuration
@@ -14,15 +14,15 @@
# Use "url = <url>" to specify the location of the rules archive to
# download. The url must begin with http://, https://, ftp://, file://
# or scp:// and end with .tar.gz or .tgz, and the file must be a
# Use "url = <url>" to specify the location of the rules archive to
# download. The url must begin with http://, https://, ftp://, file://
# or scp:// and end with .tar.gz or .tgz, and the file must be a
# gzipped tarball what contains a directory named "rules".
# You can also point to a local directory with dir://<directory>.
# Multiple "url = <url>" lines can be specified to grab multiple rules
# archives from different locations.
#
# Note: if URL is specified on the command line, it overrides all
# Note: if URL is specified on the command line, it overrides all
# possible URLs specified in the configuration file(s).
#
# The location of the official Snort rules you should use depends
@@ -30,10 +30,10 @@
# http://www.snort.org/rules/ and follow the instructions
# there to pick the right URL for your version of Snort
# (and remember to update the URL when upgrading Snort in the
# future). You can of course also specify locations to third party
# future). You can of course also specify locations to third party
# rules.
#
# As of March 2005, you must register on the Snort site to get access
# As of March 2005, you must register on the Snort site to get access
# to the official Snort rules. This will get you an "oinkcode".
# You then specify the URL as
# http://www.snort.org/pub-bin/oinkmaster.cgi/<oinkcode>/<filename>
@@ -45,7 +45,7 @@
# more information.
# URL examples follows. Replace <oinkcode> with the code you get on the
# URL examples follows. Replace <oinkcode> with the code you get on the
# Snort site in your registered user profile.
# Example for Snort
@@ -73,18 +73,18 @@
# Only OpenSSH is tested. See the FAQ for more information.
# url = scp://user@somehost.example.com:/somedir/snortrules.tar.gz
# If you use -u scp://... and need to specify a private ssh key (passed
# as -i <key> to the scp command) you can specify it here or add an
# entry in ~/.ssh/config for the Oinkmaster user as described in the
# OpenSSH manual.
# If you use -u scp://... and need to specify a private ssh key (passed
# as -i <key> to the scp command) you can specify it here or add an
# entry in ~/.ssh/config for the Oinkmaster user as described in the
# OpenSSH manual.
# scp_key = /home/oinkmaster/oinkmaster_privkey
# The PATH to use during execution. If you prefer to use external
# binaries (i.e. use_external_bins=1, see below), tar and gzip must be
# found, and also wget if downloading via ftp, http or https. All with
# optional .exe suffix. If you're on Cygwin, make sure that the path
# contains the Cygwin binaries and not the native Win32 binaries or
# The PATH to use during execution. If you prefer to use external
# binaries (i.e. use_external_bins=1, see below), tar and gzip must be
# found, and also wget if downloading via ftp, http or https. All with
# optional .exe suffix. If you're on Cygwin, make sure that the path
# contains the Cygwin binaries and not the native Win32 binaries or
# you will get problems.
# Assume UNIX style by default:
path = /bin:/usr/bin:/usr/local/bin
@@ -96,17 +96,17 @@ path = /bin:/usr/bin:/usr/local/bin
# path = /cygdrive/c/oinkmaster:/cygdrive/c/oinkmaster/bin
# We normally use external binaries (wget, tar and gzip) since they're
# already available on most systems and do a good job. If you have the
# We normally use external binaries (wget, tar and gzip) since they're
# already available on most systems and do a good job. If you have the
# Perl modules Archive::Tar, IO::Zlib and LWP::UserAgent, you can use
# those instead if you like. You can set use_external_bins below to
# choose which method you prefer. It's set to 0 by default on Win32
# (i.e. use Perl modules), and 1 on other systems (i.e. use external
# those instead if you like. You can set use_external_bins below to
# choose which method you prefer. It's set to 0 by default on Win32
# (i.e. use Perl modules), and 1 on other systems (i.e. use external
# binaries). The reason for that is that the required Perl modules
# are included on Windows/ActivePerl 5.8.1+, so it's easier to use
# those than to install the ported Unix tools. (Note that if you're
# using scp to download the archive, external scp binary is still
# used.)
# are included on Windows/ActivePerl 5.8.1+, so it's easier to use
# those than to install the ported Unix tools. (Note that if you're
# using scp to download the archive, external scp binary is still
# used.)
# use_external_bins = 0
@@ -133,7 +133,7 @@ path = /bin:/usr/bin:/usr/local/bin
# umask = 0027
# Files in the archive(s) matching this regular expression will be
# Files in the archive(s) matching this regular expression will be
# checked for changes, and then updated or added if needed.
# All other files will be ignored. You can then choose to skip
# individual files by specifying the "skipfile" keyword below.
@@ -159,10 +159,10 @@ update_files = \.rules$|\.config$|\.conf$|\.txt$|\.map$
# min_rules = 1
# By default, a basic sanity check is performed on most paths/filenames
# to see if they contain illegal characters that may screw things up.
# If this check is too strict for your system (e.g. you get bogus
# "illegal characters in filename" errors because of your local language
# By default, a basic sanity check is performed on most paths/filenames
# to see if they contain illegal characters that may screw things up.
# If this check is too strict for your system (e.g. you get bogus
# "illegal characters in filename" errors because of your local language
# etc) and you're sure you want to disable the checks completely,
# set use_path_checks to 0.
# use_path_checks = 1
@@ -174,11 +174,11 @@ update_files = \.rules$|\.config$|\.conf$|\.txt$|\.map$
# You can include other files anywhere in here by using
# "include <file>". <file> will be parsed just like a regular
# oinkmaster.conf as soon as the include statement is seen, and then
# return and continue parsing the rest of the original file. If an
# option is redefined, it will override the previous value. You can use
# as many "include" statements as you wish, and also include even more
# "include <file>". <file> will be parsed just like a regular
# oinkmaster.conf as soon as the include statement is seen, and then
# return and continue parsing the rest of the original file. If an
# option is redefined, it will override the previous value. You can use
# as many "include" statements as you wish, and also include even more
# files from included files. Example to load stuff from "/etc/foo.conf".
# include /etc/foo.conf
@@ -191,34 +191,34 @@ update_files = \.rules$|\.config$|\.conf$|\.txt$|\.map$
# or: skipfile filename1, filename2, filename3, ... #
#######################################################################
# Ignore local.rules from the rules archive by default since we might
# have put some local rules in our own local.rules and we don't want it
# to get overwritten by the empty one from the archive after each
# Ignore local.rules from the rules archive by default since we might
# have put some local rules in our own local.rules and we don't want it
# to get overwritten by the empty one from the archive after each
# update.
skipfile local.rules
# The file deleted.rules contains rules that have been deleted from
# The file deleted.rules contains rules that have been deleted from
# other files, so there is usually no point in updating it.
skipfile deleted.rules
# Also skip snort.conf by default since we don't want to overwrite our
# own snort.conf if we have it in the same directory as the rules. If
# you have your own production copy of snort.conf in another directory,
# it may be really nice to check for changes in this file though,
# especially since variables are sometimes added or modified and
# Also skip snort.conf by default since we don't want to overwrite our
# own snort.conf if we have it in the same directory as the rules. If
# you have your own production copy of snort.conf in another directory,
# it may be really nice to check for changes in this file though,
# especially since variables are sometimes added or modified and
# new/old files are included/excluded.
skipfile snort.conf
# You may want to consider ignoring threshold.conf for the same reasons
# as for snort.conf, i.e. if you customize it locally and don't want it
# to become overwritten by the default one. It may be better to put
# local thresholding/suppressing in some local file and still update
# and use the official one though, in case important stuff is added to
# You may want to consider ignoring threshold.conf for the same reasons
# as for snort.conf, i.e. if you customize it locally and don't want it
# to become overwritten by the default one. It may be better to put
# local thresholding/suppressing in some local file and still update
# and use the official one though, in case important stuff is added to
# it some day. We do update it by default, but it's your call.
# skipfile threshold.conf
# If you update from multiple URLs at the same time you may need to
# ignore the sid-msg.map (and generate it yourself if you need one) as
# If you update from multiple URLs at the same time you may need to
# ignore the sid-msg.map (and generate it yourself if you need one) as
# it's usually included in each rules tarball. See the FAQ for more info.
# skipfile sid-msg.map
@@ -243,11 +243,11 @@ skipfile snort.conf
# situations. #
# #
# Syntax: #
# modifysid SID "replacethis" | "withthis" #
# modifysid SID "replacethis" | "withthis" #
# or: #
# modifysid SID1, SID2, SID3, ... "replacethis" | "withthis" #
# or: #
# modifysid file "replacethis" | "withthis" #
# modifysid file "replacethis" | "withthis" #
# or: #
# modifysid * "replacethis" | "withthis" #
# #
@@ -289,11 +289,11 @@ skipfile snort.conf
# Example to add "tag" stuff to SID 1325.
# modifysid 1325 "sid:1325;" | "sid:1325; tag: host, src, 300, seconds;"
# Example to make SID 1378 a 'drop' rule (valid if you're running
# Example to make SID 1378 a 'drop' rule (valid if you're running
# Snort_inline).
# modifysid 1378 "^alert" | "drop"
# Example to replace first occurrence of $EXTERNAL_NET with $HOME_NET
# Example to replace first occurrence of $EXTERNAL_NET with $HOME_NET
# in SID 302.
# modifysid 302 "\$EXTERNAL_NET" | "\$HOME_NET"
@@ -306,17 +306,17 @@ skipfile snort.conf
# modifysid 1324 "(content\s*:\s*"\/bin\/sh"\s*;)" | \
# "${1} replace:"\/foo\/sh";"
# If you for some reason would like to add a comment inside the actual
# rules file, like the reason why you disabled this rule, you can do
# like this (you would normally add such comments in oinkmaster.conf
# If you for some reason would like to add a comment inside the actual
# rules file, like the reason why you disabled this rule, you can do
# like this (you would normally add such comments in oinkmaster.conf
# though).
# modifysid 1324 "(.+)" | "# 20020101: disabled this rule just for fun:\n#${1}"
# Here is an example that is actually useful. Let's say you don't care
# about incoming welchia pings (detected by SID 483 at the time of
# writing) but you want to know when infected hosts on your network
# scans hosts on the outside. (Remember that watching for outgoing
# malicious packets is often just as important as watching for incoming
# Here is an example that is actually useful. Let's say you don't care
# about incoming welchia pings (detected by SID 483 at the time of
# writing) but you want to know when infected hosts on your network
# scans hosts on the outside. (Remember that watching for outgoing
# malicious packets is often just as important as watching for incoming
# ones, especially in this case.) The rule currently looks like
# "alert icmp $EXTERNAL_NET any -> $HOME_NET any ..."
# but we want to switch that so it becomes
@@ -326,34 +326,34 @@ skipfile snort.conf
# "(.+) \$EXTERNAL_NET (.+) \$HOME_NET (.+)" | \
# "${1} \$HOME_NET ${2} \$EXTERNAL_NET ${3}"
# The wildcard (modifysid * ...) can be used to do all kinds of
# interesting things. The substitution expression will be applied on all
# matching rules. First, a silly example to replace "foo" with "bar" in
# all rules (that have the string "foo" in them, that is.)
# The wildcard (modifysid * ...) can be used to do all kinds of
# interesting things. The substitution expression will be applied on all
# matching rules. First, a silly example to replace "foo" with "bar" in
# all rules (that have the string "foo" in them, that is.)
# modifysid * "foo" | "bar"
# If you for some reason don't want to use the stream preprocessor to
# match established streams, you may want to replace the 'flow'
# If you for some reason don't want to use the stream preprocessor to
# match established streams, you may want to replace the 'flow'
# statement with 'flags:A+;' in all those rules.
# modifysid * "flow:[a-z,_ ]+;" | "flags:A+;"
# Example to convert all rules of classtype attempted-admin to 'drop'
# Example to convert all rules of classtype attempted-admin to 'drop'
# rules (for Snort_inline only, obviously).
# modifysid * "^alert (.*classtype\s*:\s*attempted-admin)" | "drop ${1}"
# This one will append some text to the 'msg' string for all rules that
# This one will append some text to the 'msg' string for all rules that
# have the 'tag' keyword in them.
# modifysid * "(.*msg:\s*".+?)"(\s*;.+;\s*tag:.*)" | \
# "${1}, going to tag this baby"${2}"
# There may be times when you want to replace multiple occurrences of a
# certain keyword/string in a rule and not just the first one. To
# replace the first two occurrences of "foo" with "bar" in SID 100,
# There may be times when you want to replace multiple occurrences of a
# certain keyword/string in a rule and not just the first one. To
# replace the first two occurrences of "foo" with "bar" in SID 100,
# simply repeat the modifysid statement:
# modifysid 100 "foo" | "bar"
# modifysid 100 "foo" | "bar"
# Or you can even specify a SID list but repeat the same SID as many
# Or you can even specify a SID list but repeat the same SID as many
# times as required, like:
# modifysid 100,100,100 "foo" | "bar"

14
roles/Sharingan/files/rkhunter/rkhunter.conf
View File

@@ -90,7 +90,7 @@
#
# If the mirrors file is read-only, then the '--versioncheck' command-line
# option can only be used if this option is set to '0'.
#
#
# The default value is '1'.
#
#ROTATE_MIRRORS=1
@@ -635,7 +635,7 @@ ALLOWHIDDENDIR=/dev/.udev
# This option may be specified more than once, and may use wildcard characters.
#
# The default value is the null string.
#
#
#ALLOWHIDDENFILE=/usr/share/man/man1/..1.gz
#ALLOWHIDDENFILE=/usr/bin/.fipscheck.hmac
#ALLOWHIDDENFILE=/usr/bin/.ssh.hmac
@@ -907,7 +907,7 @@ ALLOWHIDDENDIR=/dev/.udev
#
#APP_WHITELIST=""
#
#
# Set this option to scan for suspicious files in directories which pose a
# relatively higher risk due to user write access.
#
@@ -918,7 +918,7 @@ ALLOWHIDDENDIR=/dev/.udev
#
# Please consider adding all directories the user the (web)server runs as,
# and has write access to, including the document root (e.g: '/var/www') and
# log directories (e.g: '/var/log/httpd').
# log directories (e.g: '/var/log/httpd').
#
# This is a space-separated list of directory pathnames. The option may be
# specified more than once.
@@ -968,7 +968,7 @@ ALLOWHIDDENDIR=/dev/.udev
#
# The following options can be used to whitelist network ports which are known
# to have been used by malware.
# to have been used by malware.
#
# The PORT_WHITELIST option is a space-separated list of one or more of two
# types of whitelisting. These are:
@@ -1233,10 +1233,10 @@ ALLOWHIDDENDIR=/dev/.udev
#
# You should only activate this feature as part of a more thorough
# investigation, which should be based on relevant best practices and
# procedures.
# procedures.
#
# Enabling this feature implies you have the knowledge to interpret the
# results properly.
# results properly.
#
# The default value is the null string.
#

2
roles/Sharingan/files/suricata/suricata.yaml
View File

@@ -134,7 +134,7 @@ outputs:
# Header name were the actual IP address will be reported, if more than
# one IP address is present, the last IP address will be the one taken
# into consideration.
header: X-Forwarded-For
header: X-Forwarded-For
# a line based log of HTTP requests (no alerts)
- http-log:

12
roles/Sharingan/tasks/data.yml
View File

@@ -1,10 +1,10 @@
---
---
- name: Sharingan data packages
become: yes
package:
state: present
name:
name:
- syslog-ng
- monit
- monitoring-plugins
@@ -18,13 +18,13 @@
- name: Sharingan-Data include apps dir
become: yes
register: base_config
lineinfile:
lineinfile:
path: /etc/syslog-ng/syslog-ng.conf
line: "{{ item }}"
loop:
loop:
- '# Allow compartmentalization of config'
- '@include "apps.d/*.conf"'
- name: Sharingan-Data conf
become: yes
template:

4
roles/Sharingan/tasks/heartbeat.yml
View File

@@ -9,9 +9,9 @@
owner: root
group: root
mode: 0750
loop:
loop:
- sharingan-heartbeat.timer
- sharingan-heartbeat.service
- sharingan-heartbeat.service
- name: Sharingan-Data heartbeat timer
become: yes

12
roles/Sharingan/tasks/ids.yml
View File

@@ -4,7 +4,7 @@
become: yes
register: package_install
package:
name:
name:
- sshguard
- suricata
- oinkmaster
@@ -45,7 +45,7 @@
- name: Copy rkhunter service
register: rkhunter_service
become: yes
loop:
loop:
- rkhunter.service
- rkhunter.timer
copy:
@@ -69,7 +69,7 @@
- name: Copy oinkmaster service
register: oinkmaster_service
become: yes
loop:
loop:
- oinkmaster.service
- oinkmaster.timer
copy:
@@ -88,7 +88,7 @@
become: yes
when: package_install.changed or oinkmaster_conf.changed
service:
name: oinkmaster.service
name: oinkmaster.service
state: started
- name: Update rkhunter DB
@@ -98,12 +98,12 @@
- name: IDS services
become: yes
loop:
loop:
- suricata.service
- sshguard.service
- oinkmaster.timer
- rkhunter.timer
service:
service:
name: "{{ item }}"
state: restarted
enabled: yes

4
roles/Sharingan/tasks/main.yml
View File

@@ -1,5 +1,5 @@
---
---
- import_tasks: ../roles/Sharingan/tasks/data.yml
- import_tasks: ../roles/Sharingan/tasks/eval.yml
- import_tasks: ../roles/Sharingan/tasks/heartbeat.yml

4
roles/Sharingan/tasks/scans.yml
View File

@@ -24,7 +24,7 @@
become: yes
register: lynis_svc
copy:
src: "lynis/{{ item }}"
src: "lynis/{{ item }}"
dest: /usr/lib/systemd/system/
owner: root
group: root
@@ -37,7 +37,7 @@
become: yes
register: clam_svc
copy:
src: "clamav/{{ item }}"
src: "clamav/{{ item }}"
dest: /usr/lib/systemd/system/
owner: root
group: root

2
roles/Sharingan/tasks/siem.yml
View File

@@ -16,7 +16,7 @@
- mongodb
- graylog
service:
name: "{{ item }}"
name: "{{ item }}"
state: started
enabled: yes

3
roles/Sharingan/templates/monitrc.j2
View File

@@ -153,7 +153,8 @@ set log syslog
set httpd port 2812 and
use address localhost # only accept connection from localhost (drop if you use M/Monit)
allow localhost # allow localhost to connect to the server and
allow admin:"{{ monitcli | default('monit') }}" # require user 'admin' with password 'monit'
# require user 'admin' with password
allow admin:"{{ secrets[Sharingan][monit] | default('monit') }}"
#with ssl { # enable SSL/TLS and set path to server certificate
# pemfile: /etc/ssl/certs/monit.pem
#}