Fixing Nazara errors

bin/generate-pihole-dns-dhcp.py

@@ -12,8 +12,9 @@ import os
 import sys
 import yaml
 
-dnsfilepath="roles/Nazara/files/dns"
-dhcpfilepath="roles/Nazara/files/dhcp"
+rolepath='../roles/Nazara/files'
+dnsfilepath=rolepath+"/dns"
+dhcpfilepath=rolepath+"/dhcp"
 
 def WriteDHCPEntry(content,hosttype,hostclass):
     ### Create the DHCP entry
@@ -25,7 +26,7 @@ def WriteDHCPEntry(content,hosttype,hostclass):
     with open(dhcpfilepath,'a') as dhcpfile: 
         for host in content['all']['children'][hosttype]['children'][hostclass]['hosts']:
             try:
-                dhcpfile.write('dhcp-host=' + content['all']['children'][hosttype]['children'][hostclass]['hosts'][host]['vars']['mac'] + ',' + content['all']['children'][hosttype]['children'][hostclass]['hosts'][host]['vars']['ip'] + ',' + host + '.' + content['all']['vars']['replica_domain'] + '\n')
+                dhcpfile.write('dhcp-host=' + content['all']['children'][hosttype]['children'][hostclass]['hosts'][host]['mac'] + ',' + content['all']['children'][hosttype]['children'][hostclass]['hosts'][host]['ip'] + ',' + host + '.' + content['all']['vars']['replica_domain'] + '\n')
             except:
                 print(host + ' is not complete for DHCP.')
 
@@ -39,7 +40,7 @@ def WriteDNSEntry(content,hosttype,hostclass):
     with open(dnsfilepath,'a') as dnsfile: 
         for host in content['all']['children'][hosttype]['children'][hostclass]['hosts']:
             try:
-                dnsfile.write(content['all']['children'][hosttype]['children'][hostclass]['hosts'][host]['vars']['ip'] + ' ' + host + '.' + content['all']['vars']['replica_domain'] + ' ' + host + '\n')
+                dnsfile.write(content['all']['children'][hosttype]['children'][hostclass]['hosts'][host]['ip'] + ' ' + host + '.' + content['all']['vars']['replica_domain'] + ' ' + host + '\n')
             except:
                 print(host + ' is not complete for DNS.')
 
@@ -48,6 +49,9 @@ def GenerateFiles(file):
     # param file: the file to work on
     global dnsfile
 
+    if not os.path.isdir(rolepath):
+        os.mkdir(rolepath)
+
     # Parse the yaml
     with open(file, 'r') as stream:
         content = yaml.safe_load(stream)
@@ -55,7 +59,6 @@ def GenerateFiles(file):
         # Clear the DNS file
         with open(dhcpfilepath,'w') as dhcpfile:
             dhcpfile.write('dhcp-range='+content['all']['vars']['dhcprange']+'\n')
-            dhcpfile.write('dhcp-option=option:router,'+content['all']['vars']['router']+'\n')
             dhcpfile.write('dhcp-option=option:dns-server,'+content['all']['vars']['dns']+'\n\n')
             dhcpfile.write('dhcp-range='+content['all']['vars']['staticrange']+'\n')
         with open(dnsfilepath,'w') as dnsfile:
@@ -63,7 +66,7 @@ def GenerateFiles(file):
 
         # Add DNS entries for each host
         hosttype = 'managed'
-        for hostclass in ['physical','virtual','geth_hubs']: 
+        for hostclass in ['physical','virtual','geth_hubs']:
             WriteDNSEntry(content,hosttype,hostclass)
             WriteDHCPEntry(content,hosttype,hostclass)
         hosttype = 'unmanaged'

roles/Cyberbrain/README.md

@@ -1,4 +1,4 @@
-Cyberbrain is a way to ensure that so long as a person is connected to the Internet and authorized, they're able to connect to, use, and control the AniNIX. It's a web-based shell emulator for connecting to the system.
+Cyberbrain is a way to ensure that so long as a person is connected to the Internet and authorized, they're able to connect to, use, and control the AniNIX. It's a web-based shell emulator for connecting to the system. It can serve as an alternative to using the [Terminal & SSH add-on](https://www.home-assistant.io/common-tasks/supervised/#installing-and-using-the-ssh-add-on-requires-enabling-advanced-mode-for-the-ha-user) for [AniNIX/Geth](../Geth/) in cases where a separate security posture is needed for each.
 
 **Warning**: This is a fallback measure -- browsers are still inherently less secure than hard clients like [Git Bash](https://git-scm.com/download/win) or [OpenSSH](https://www.openssh.com/portable.html).
 

roles/Grimoire/README.md

@@ -8,9 +8,9 @@ Grimoire has a user, postgres, with a home directory of `/var/lib/postgres/`. Th
 
 ## Backups
 Backups are provided by [AniNIX/Aether](../Aether). They can be restored with the following:
-<pre>
+```
 psql -U dbuser -d db -f backup.sql
-</pre>
+```
 
 # Available Clients
 There are no clients for the Grimoire -- Singularity and Wiki maintain their tables.

roles/Nazara/files/pihole-FTL.conf

@@ -0,0 +1,2 @@
+PRIVACYLEVEL=0
+RATE_LIMIT=1000/5

roles/Nazara/tasks/main.yml

@@ -9,11 +9,17 @@
 
  - name: Install pi-hole if needed
    become: yes
+   register: pihole_install
    command: 
        creates: /usr/bin/pihole-FTL
-       cmd: bash basic-install.sh
+       cmd: false # bash basic-install.sh
        chdir: '/opt/pi-hole/automated install'
 
+ - name: Ensure pihole web admin password
+   become: yes
+   command: "pihole -a -p {{ passwords['Nazara'] }}"
+     # when: pihole_install.changed
+
  - name: Generate DNS/DHCP from inventory
    delegate_to: localhost
    run_once: true
@@ -29,11 +35,6 @@
      group: pihole
      mode: 0644
 
- - name: Reload dns
-   become: yes
-   command: "pihole restartdns"
-   when: dns_updated.changed
-
  - name: Nazara DHCP
    become: yes
    register: dhcp_updated
@@ -44,8 +45,36 @@
      group: root
      mode: 0644
 
+ - name: Nazara Configuration
+   become: yes
+   register: conf_updated
+   copy:
+     src: pihole-FTL.conf
+     dest: /etc/pihole/pihole-FTL.conf
+     owner: root
+     group: root
+     mode: 0644
+
+
+ - name: Nazara DHCP Leases dir
+   become: yes
+   file:
+     path: /var/lib/misc/
+     state: directory
+     owner: root
+     group: root
+     mode: 0777
+
+ - name: Nazara DHCP Leases
+   become: yes
+   file:
+     path: /var/lib/misc/dnsmasq.leases
+     state: touch
+     owner: pihole
+     group: pihole
+     mode: 0660
+
  - name: Reload services
    become: yes
    command: pihole restartdns
-   when: dns_updated.changed or dhcp_updated.changed
-
+   when: dns_updated.changed or dhcp_updated.changed or conf_updated.changed

roles/SSH/README.md

@@ -1,18 +1,17 @@
-Remote access is important in the AniNIX, and so we support the use of the [https://wiki.archlinux.org/index.php/Secure_Shell OpenSSH] protocol via [[ShadowArch]] to supporting hosts.
+Remote access is important in the AniNIX, and so we support the use of the [OpenSSH](https://wiki.archlinux.org/index.php/Secure_Shell) protocol to supporting hosts.
 
 # Etymology
-SSH is named for the protocol on which it's built.
+SSH is named for the protocol on which it's built. It's so ubiquitous that we don't rename it.
 
 # Relevant Files and Software
-Most of this service's configuration lives in [file:///etc/ssh/sshd_config sshd_config]. This includes match statements on what groups are allowed to connect, allowed protocols, and somewhat importantly the ForceCommand directives that hold certain users captive to specific operations.
+Most of this service's configuration lives in [sshd_config](files/sshd_config) as specified in [sshd_config(5)](https://man.archlinux.org/man/core/openssh/sshd_config.5.en). This includes match statements on what groups are allowed to connect, allowed protocols, and somewhat importantly the ForceCommand directives that hold certain users captive to specific operations.
 
 VNC and X11 forwarding can be used over SSH to allow graphical clients. X11 forwarding without SSH compression is generally slower. To allow VNC, log in over SSH and forward remote port 5901 to localhost port 5901. Start the VNC server on the remote, and use a VNC viewer like tightVNC portable to view the remote desktop.
 
+This role does expect that you have a public key in your `.ssh` folder named `deploy.pub`. This public key will be put on all servers, and as such it is intrinsically necessary that there be a passphrase on the private key to protect it from compromise. [AniNIX/ShadowArch](/AniNIX/ShadowArch) will provide a convenient [service file](/AniNIX/ShadowArch/src/branch/main/EtcFiles/ssh-agent@.service) to wrap the ssh-agent service for you to make working with this key easier.
+
 # Available Clients
-* Windows users should use [http://www.putty.org/ PuTTY]. The AniNIX considers this important enough that a copy of PuTTY is mirrored in [https://aninix.net/wolfpack/ WolfPack].[[Category:CachedClient]]
 * Mac has a native client in their Terminal application.
-* Linux users can install [https://wiki.archlinux.org/index.php/Secure_Shell openssh].
-* Android users can use [https://serverauditor.com/ Server Auditor].
-}}
-[[Category:Public_Service]]
-[[Category:LDAP]]
+* Windows users should use [Git Bash](https://git-scm.com/download/win).
+* Linux users can install [openssh](https://archlinux.org/packages/core/x86_64/openssh/).
+* Android users can use [AdminHands](https://play.google.com/store/apps/details?id=com.arpaplus.adminhands).

roles/Sharingan/README.md

@@ -29,6 +29,26 @@ TODO
 
 ## Monit
 
+## Graylog
+
+## Elasticsearch
+Elasticsearch acts as graylog's data backend. 
+
+We have seen issues where poor disk i/o or unplanned shutdown can cause Elasticsearch to have index corruption. 
+
+1. Stop elasticsearch
+1. From `/usr/share/elasticsearch/lib`, you can use `java -cp lucene-core*.jar -ea:org.apache.lucene... org.apache.lucene.index.CheckIndex /usr/share/elasticsearch/data/nodes/0/indices/1nJc43t7TGuHmVR3Q5w9PA/1/index -verbose -exorcise` (on the right index) to exorcise the corrupted data. 
+1. Remove corruption flags: `rm /usr/share/elasticsearch/data/nodes/0/indices/1nJc43t7TGuHmVR3Q5w9PA/1/index/corrupted_*`
+1. Restart elasticsearch
+1. Retry shard allocation:
+```
+curl -X POST http://127.0.0.1:9200/_cluster/reroute?retry_failed=true
+curl -XGET localhost:9200/_cluster/allocation/explain?pretty
+```
+
+## Mongodb
+MongoDB holds the graylog config for us.
+
 # Available Clients
 See [[WebServer#Available Clients|AniNIX::Webserver's client list]].
 

roles/WolfPack/README.md

@@ -0,0 +1,14 @@
+WolfPack is a webcrawler for the AniNIX. Public results from Core's instance will be available from [https://wolfpack.aninix.net/wolfpack the WebServer] -- this may be locked to admins, for reproducibility reasons.
+
+Note: Code for this service is encoded in [the WolfPack repo](/AniNIX/WolfPack) rather than here -- we just include the package.
+
+# Etymology
+WolfPack is named for its operation. "Pups" live on disk as .pup files -- these will grow up and retrieve the results that feed the system. An alpha sends pack members to raise a pup and collect the results for the pack. This role will update configuration to [the configuration directory](file:///usr/local/etc/WolfPack).
+
+## VPN protection and Offloading.
+Some countries and areas take issue with some searches and downloads. As such, the offload-wolfpack executable will allow a [DarkNet](../DarkNet) service, deployed on a unique host, to merge results. In your Ansible inventory, set the wolfpack_service YAML variable for the host to `offload-wolfpack@somehost.timer` to enable that service instead of the normal wolfpack.timer.
+
+This requires SSH keys to be set up between the offloading hosts and the target location, but this will run some version of wolfpack and send the results to the target. This is helpful for a server like Core that requires network uptime and stable external accessibility but needs VPN functionality for anonymity. This requires significant user intervention and customization -- this option is provided as a stub. 
+
+## Alternatives
+Google Alerts can provide an alternative to the Wolfpack's search pup type. Downloads can be done manually, and some torrent clients will have search and queuing options.

roles/Yggdrasil/package/README.md

@@ -0,0 +1,8 @@
+This is a collection of scripts we use for managing yggdrasil data.
+
+1. yggdrasil-get: API for pulling data into Yggdrasil.
+1. yggdrasil-lock: API for setting permissions safely.
+1. yggdrasil-set-music-data: API for updating a music file with the new detected metadata from the path. Assumes `/srv/yggdrasil/Music/$genre/$artist/$album`.
+1. yggdrasil-sha256: Get a SHA-256 hash of the current library. This is good for checking media changes over time in conjunction with [AniNIX/Aether](/AniNIX/Aether). 
+1. yggdrasil-sort-shows: Look at `/srv/yggdrasil/new_acquisition` and try to find the right folder in `/srv/yggdrasil/Videos/Shows` to stash it in. Will try to put it under the show name and the season.
+1. yggdrasil-unlock: API for allowing writes to media.

roles/common/README.md

@@ -0,0 +1 @@
+This role is only intended as a library of handlers to be shared between roles in this project.