Refactor for ActiveRecord and Config Files

Update Custom Commands to leverage ActiveRecord and save commands to a
database. By default, it will use SQLite. Configuration for this (and
potential configuration for PostgreSQL and MySQL) live in
`config/db.yml`. Inculde a Rakefile for handling DB creation and
migrations.

Rakefile: Add Rakefile to handle running the bot, and DB management
Gemfile: Update with new gem dependencies
db/migrate/*: ActiveRecord migrations for Custom Command
custom_commands.rb: Update to leverage ActiveRecord

Leverage the Rakefile to start the bot, removing the binary file. Update
the Dockerfile to also leverage the Rakefile.

Dockerfile: Update to use Rakefile, and install new dependencies
chronicle: Remove unnecessary start file

Refactor the `chronicle_bot` file into `chronicle` and `matrix`

chronicle.rb: General Chronicle setup
matrix.rb: Start a Matrix-specific bot

Update the bot to read configuration from files, instead of either the
environment, or hard-coded values.

config/db.yml: Database configuration
config/bot.yml: General bot configuration

Update the README to reflect the above change with regards to running
the bot either using the Rakefile, or using a Docker container.

Dockerfile

@@ -1,6 +1,7 @@
 FROM ruby:2.7-alpine
 
 RUN bundle config --global frozen 1
+RUN apk add build-base sqlite sqlite-dev sqlite-libs
 
 WORKDIR /app
 
@@ -9,4 +10,4 @@ RUN bundle install
 
 COPY . .
 
-CMD ["./chronicle"]
+CMD ["bundle", "exec", "rake", "chronicle:start"]

Gemfile

@@ -3,3 +3,9 @@
 source 'https://rubygems.org'
 gem 'matrix_sdk', '~> 2.0'
 gem 'faraday', '~> 1.0'
+
+gem "activerecord", "~> 6.1"
+
+gem "rake", "~> 13.0"
+
+gem "sqlite3", "~> 1.4"

README.md

@@ -37,27 +37,27 @@ You can run your own instance of Chronicle with a few steps:
 
 1. Fork the repository, and clone it locally
 2. Setup a bot user in Matrix, and get its "Access Token"
-3. Export the access token to `CHRONICLE_ACCESS_TOKEN`
+3. Update `config/bot.yml` with the Homeserver URL, and the Access Token
-4. Export your Matrix homeserver URL to `CHRONICLE_HOMESERVER`
+4. Update `config/db.yml` with any desired changes (defaults use SQLite)
-5. (Optional) Set `CHRONICLE_DEBUG` to 1 to get debug output
+5. (Optional) Update `config/bot.yml` with any additional changes
 6. Run `bundle update` to install dependencies
-7. Run `bundle exec chronicle`
+7. Run `rake chronicle:start`
 8. Invite the bot user to a room, and `!ping` to make sure it's working!
 
 # Docker
 
 The included Dockerfile is very simplistic, and may be expanded in the future.
 For now, there is no pre-built image stored in a Hub, so you'll need to build
-your own. From the project directory, `docker build -t chronicle-bot .`
+your own.
 
-Export the access token to `CHRONICLE_ACCESS_TOKEN`
+1. Fork the repository, and clone it locally
-Export your Matrix homeserver URL to `CHRONICLE_HOMESERVER`
+2. Setup a bot user in Matrix, and get its "Access Token"
-(Optional) Set `CHRONICLE_DEBUG` to 1 to get debug output
+3. Update `config/bot.yml` with the Homeserver URL, and the Access Token
-
+4. Update `config/db.yml` with any desired changes (defaults use SQLite)
-Then, you can run Chronicle in Docker:
+5. (Optional) Update `config/bot.yml` with any additional changes
-
+6. Build the image: `docker build -t chronicle-bot .`
-`docker run --rm --name chronicle -e CHRONICLE_HOMESERVER -e
+7. Run Chronicle in Docker with `docker run --rm --name chronicle chronicle-bot`
-CHRONICLE_ACCESS_TOKEN chronicle-bot`
+8. Invite the bot user to a room, and `!ping` to make sure it's working!
 
 # Contribute
 

Rakefile

@@ -0,0 +1,59 @@
+namespace :chronicle do
+  desc 'Start the bot'
+  task :start do
+    require 'active_record'
+    require_relative 'lib/chronicle'
+
+    db_config = YAML::load(File.open('config/database.yml'))
+    ActiveRecord::Base.establish_connection(db_config)
+
+    bot_config = YAML::load(File.open('config/bot.yml'))
+    Chronicle::Config.load_config(bot_config)
+
+    Chronicle.start
+  end
+end
+
+namespace :db do
+  require 'active_record'
+  require 'yaml'
+
+  Dir[File.join(__dir__, 'db', 'migrate', '*.rb')].each do |file|
+    require file
+  end
+
+  task :connect do
+    connection_details = YAML::load(File.open('config/database.yml'))
+    ActiveRecord::Base.establish_connection(connection_details)
+  end
+
+  desc "Create a new database"
+  task :create do
+    connection_details = YAML::load(File.open('config/database.yml'))
+
+    if connection_details["adapter"] == 'sqlite3'
+      if File.exists?(connection_details["database"])
+        puts 'DB already exists' 
+      else
+        File.open(connection_details["database"], 'w+') {}
+      end
+    else
+      ActiveRecord::Base.establish_connection(connection_details)
+      ActiveRecord::Base.connection.create_database(
+        connection_details["database"]
+      )
+    end
+  end
+
+  desc "Run the migrations"
+  task :migrate => 'db:connect' do
+    # ActiveRecord::MigrationContext.new('db/migrate/').migrate()
+    CreateCustomCommands.migrate(:up)
+  end
+
+  desc "Clear the database"
+  task :drop => 'db:connect' do
+    # ActiveRecord::Migration.migrate(:down)
+    CreateCustomCommands.migrate(:down)
+  end
+end

chronicle

@@ -1,5 +0,0 @@
-#!/usr/bin/env ruby
-
-require_relative 'lib/chronicle_bot'
-
-Chronicle::Matrix.start

config/bot.yml

@@ -0,0 +1,7 @@
+matrix:
+  homeserver: ''
+  token: ''
+
+log:
+  file: 'chronicle.log'
+  debug: 1

config/db.yml

@@ -0,0 +1,19 @@
+# SQLite Configuration
+adapter:  'sqlite3'
+database: 'db/chronicle-bot.db'
+
+# PostgreSQL Configuration
+# adapter:  'postgresql'
+# database: 'chronicle-bot'
+# host:     'localhost:1234'
+# username: 'chronicle'
+# password: 'ch4ng3m3'
+# encoding: 'utf8'
+
+# MySQL Configuration
+# adapter:  'mysql2'
+# database: 'chronicle-bot'
+# host:     'localhost:1234'
+# username: 'chronicle'
+# password: 'ch4ng3m3'
+# encoding: 'utf8'

db/migrate/001_create_custom_commands.rb

@@ -0,0 +1,14 @@
+class CreateCustomCommands < ActiveRecord::Migration[5.2]
+  def up
+    create_table :custom_commands do |table|
+      table.string :roomid
+      table.string :command
+      table.string :response
+      table.timestamps
+    end
+  end
+
+  def down
+    drop_table :custom_commands
+  end
+end

lib/addons/custom_commands.rb

@@ -5,6 +5,7 @@
 # For example, "!hello" might print:
 #   "Welcome to the channel! You can follow [this link](link) to visit
 #   our website!"
+require 'active_record'
 require 'json'
 
 module Chronicle
@@ -19,10 +20,8 @@ module Chronicle
 
       def initialize(bot)
         @bot = bot
-        @msgid = 'tmp_until_per-room'
-        @custom_commands = read_commands(@msgid)
 
-        @bot.available_commands(self, @custom_commands.keys)
+        @bot.available_commands(self, list_commands)
       end
 
       # Provide help for the commands of this addon
@@ -49,6 +48,7 @@ module Chronicle
       # @param message [Message object] The relevant message object
       def matrix_command(message)
         pfx = @bot.cmd_prefix
+        roomid = message.room_id
         cmd = message.content[:body].split(/\s+/)[0].gsub(/#{pfx}/, '')
         msgstr = message.content[:body]
                         .gsub(/#{pfx}\w+\s*/, '')
@@ -58,43 +58,34 @@ module Chronicle
 
         case cmd
         when "addcommand"
-          res = handle_addcommand(msgstr)
+          res = handle_addcommand(roomid, msgstr)
         when "modcommand"
-          res = handle_modcommand(msgstr)
+          res = handle_modcommand(roomid, msgstr)
         when "remcommand"
-          res = handle_remcommand(msgstr)
+          res = handle_remcommand(roomid, msgstr)
         else
-          res = runcmd(cmd)
+          res = handle_runcommand(roomid, cmd)
         end
 
-        room = @bot.client.ensure_room(message.room_id)
+        room = @bot.client.ensure_room(roomid)
 
         room.send_notice(res)
       end
 
-      # Add a new custom command
-      def addcmd(message)
-        command = message.slice!(/\w+\s+/).strip
-
-        return cmd_add_error(command) if verify_commands(command)
-        return cmd_addon_error if addon_command(command)
-
-        @custom_commands[command] = message
-        @bot.available_commands(self, [command])
-        save_commands(@msgid)
-
-        "New command saved: !#{command}"
-      end
-
       # Adds a new custom command
       #
+      # @param roomid [string] The Matrix Room ID
       # @param message [String] The command plus response
-      # @return 
+      # @return A response message
-      def handle_addcommand(message)
+      def handle_addcommand(roomid, message)
-        res = 'Usage: !addcommand NAME RESPONSE'
+        res = cmd_add_usage
 
         if message.split(/\s+/).count > 1
-          res = addcmd(message)
+          command = message.slice!(/\w+\s+/).strip
+
+          res = save_command(roomid, command, message)
+
+          @bot.available_commands(self, [command])
         end
 
         res
@@ -102,12 +93,18 @@ module Chronicle
 
       # Modify an existing custom command
       #
+      # @param roomid [string] The Matrix Room ID
       # @param message [hash] The message data from Matrix
-      def handle_modcommand(message)
+      # @return A response message
-        res = 'Usage: !modcommand NAME NEW-RESPONSE'
+      def handle_modcommand(roomid, message)
+        res = cmd_add_usage
 
         if message.split(/\s+/).count > 1
-          res = modcmd(message)
+          command = message.slice!(/\w+\s+/).strip
+
+          res = mod_command(roomid, command, message)
+
+          @bot.available_commands(self, [command])
         end
 
         res
@@ -115,47 +112,37 @@ module Chronicle
 
       # Remove an existing custom command
       #
+      # @param roomid [string] The Matrix Room ID
       # @param message [hash] The message data from Matrix
-      def handle_remcommand(message)
+      # @return A response message
-        res = 'Usage: !remcommand NAME'
+      def handle_remcommand(roomid, message)
+        res = cmd_rem_usage
 
         if message.split(/\s+/).count == 1
-          res = remcmd(message)
+          command = message.strip
+
+          res = remove_command(roomid, command)
+
+          @bot.disable_commands(command)
         end
 
         res
       end
 
-      # Modify an existing custom command
+      # Return the response for a custom command
-      def modcmd(message)
+      #
-        command = message.slice!(/\w+\s+/).strip
+      # @param roomid [string] The Matrix Room ID
+      # @param message [hash] The message data from Matrix
+      # @return A response message
+      def handle_runcommand(roomid, message)
+        res = cmd_rem_usage
 
-        return cmd_mod_error(command) unless verify_commands(command)
+        res = CustomCommands.find_by(
+          roomid: roomid,
+          command: message.strip
+        ).response
 
-        @custom_commands[command] = message
+        res
-        save_commands(@msgid)
-
-        "!#{command} modified."
-      end
-
-      # Delete an existing custom command
-      def remcmd(message)
-        command = message.strip
-
-        return cmd_rem_error unless verify_commands(command)
-
-        @custom_commands.delete(command)
-        @bot.disable_commands(command)
-        save_commands(@msgid)
-
-        "!#{command} removed."
-      end
-
-      # Execute a custom command
-      def runcmd(command)
-        return cmd_mod_error(command) unless verify_commands(command)
-
-        @custom_commands[command]
       end
 
       private
@@ -165,12 +152,6 @@ module Chronicle
         @bot.all_commands.keys.include?(command)
       end
 
-      # Error message when trying to add an existing command
-      def cmd_add_error(command)
-        'This custom command already exists. '\
-        "You can modify it by typing `!modcommand #{command}`"
-      end
-
       # Help message for addcommand
       def cmd_add_usage
         'Add a custom command. '\
@@ -212,33 +193,84 @@ module Chronicle
         "\nUsage: !remcommand EXISTING-COMMAND"
       end
 
-      # Read the existing saved commands into memory
+      # List all available commands from the DB
-      def read_commands(msgid)
+      def list_commands
-        cmds = {}
+        commands = CustomCommands.select(:command).map do |c|
-        cmds_file = "#{msgid}_custom_commands.json"
+          c.command
-
-        if File.exist?(cmds_file) && !File.empty?(cmds_file)
-          File.open(cmds_file, 'r') do |f|
-            cmds = JSON.parse(f.read)
-          end
         end
 
-        cmds
+        commands
       end
 
-      # Save the existing commands to a local file
+      # Modify an existing command in the DB
-      def save_commands(msgid)
+      def mod_command(roomid, command, response)
-        cmds_file = "#{msgid}_custom_commands.json"
+        res = "Command updated: !#{command}"
 
-        File.open(cmds_file, 'w') do |f|
+        cc = CustomCommands.find_by(:command => command)
-          f.write(@custom_commands.to_json)
+        cc.response = response
+
+        unless cc.save
+          @bot.scribe.info('CustomCommander') {
+            "Problem modifying: #{command}. Not saved." 
+          }
+
+          return cc.errors.objects.first.full_message
         end
+        
+        @bot.scribe.info('CustomCommander') {
+          "Custom command updated: #{command}"
+        }
+
+        res
       end
 
-      # Check if a command already exists
+      # Remove an existing command from the DB
-      def verify_commands(command)
+      def remove_command(roomid, command)
-        @custom_commands.keys.include?(command)
+        res = "Command removed: !#{command}"
+
+        CustomCommands.find_by(
+          roomid: roomid,
+          command: command
+        ).delete
+
+        res
       end
+
+      # Save a new command to the DB
+      def save_command(roomid, command, response)
+        res = "Command saved: !#{command}"
+
+        cc = CustomCommands.new do |c|
+          c.roomid = roomid
+          c.command = command
+          c.response = response
+        end
+
+        unless cc.save
+          @bot.scribe.info('CustomCommander') {
+            "Duplicate command: #{command}. Not saved." 
+          }
+
+          return cc.errors.objects.first.full_message
+        end
+        
+        @bot.scribe.info('CustomCommander') {
+          "Custom command saved: #{command}"
+        }
+
+        res
+      end
+    end
+
+    # The ActiveRecord model for handling the custom commands
+    class CustomCommands < ActiveRecord::Base
+      validates_presence_of :roomid, :command, :response
+      validates_length_of :command, { minimum: 1 }
+      validates_length_of :response, { minimum: 1 }
+
+      validates :command, uniqueness: { message:
+        "already exists. You can modify it by typing `!modcommand %{value}`"
+      }
     end
   end
 end

lib/chronicle.rb

@@ -0,0 +1,61 @@
+# frozen_string_literal: true
+
+require 'active_record'
+require 'faraday'
+require 'json'
+require 'logger'
+require 'matrix_sdk'
+require 'yaml'
+
+require_relative './matrix'
+
+# Require any addons
+Dir[File.join(__dir__, 'addons', '*.rb')].each do |file|
+  require file
+end
+
+# Chronicle Bot
+module Chronicle
+  # A filter to simplify syncs
+  BOT_FILTER = {
+    presence: { types: [] },
+    account_data: { types: [] },
+    room: {
+      ephemeral: { types: [] },
+      state: {
+        types: ['m.room.*'],
+        lazy_load_members: true
+      },
+      timeline: {
+        types: ['m.room.message']
+      },
+      account_data: { types: [] }
+    }
+  }.freeze
+
+  # Establish configuration for Chronicle
+  module Config
+    class << self
+      # Matrix connection configuration attributes
+      attr_accessor :matrix_homeserver, :matrix_access_token
+
+      # Logging configuration attributes
+      attr_accessor :log_file, :log_verbose
+    end
+
+    # Load a configuration Hash, and store in "module variables"
+    #
+    # @param config [Hash] a configuration hash
+    def self.load_config(config)
+      @matrix_homeserver = config["matrix"]["homeserver"]
+      @matrix_access_token = config["matrix"]["token"]
+
+      @log_file = config["log"]["file"]
+      @log_verbose = config["log"]["debug"]
+    end
+  end
+
+  def self.start()
+    Chronicle::Matrix.start
+  end
+end

lib/matrix.rb

@@ -2,6 +2,7 @@
 
 require 'faraday'
 require 'json'
+require 'logger'
 require 'matrix_sdk'
 
 # Require any addons
@@ -11,51 +12,24 @@ end
 
 # Chronicle Bot
 module Chronicle
-  # A filter to simplify syncs
-  BOT_FILTER = {
-    presence: { types: [] },
-    account_data: { types: [] },
-    room: {
-      ephemeral: { types: [] },
-      state: {
-        types: ['m.room.*'],
-        lazy_load_members: true
-      },
-      timeline: {
-        types: ['m.room.message']
-      },
-      account_data: { types: [] }
-    }
-  }.freeze
-
   # Chronicle Bot for Matrix
   module Matrix
     # Begin the beast
     def self.start
-      unless ENV["CHRONICLE_HOMESERVER"]
+      if Chronicle::Config.log_verbose >= 1
-        raise "Export your homeserver URL to CHRONICLE_HOMESERVER"
-      end
-
-      unless ENV["CHRONICLE_ACCESS_TOKEN"]
-        raise "Export your access token to CHRONICLE_ACCESS_TOKEN"
-      end
-
-      if ENV["CHRONICLE_DEBUG"]
         Thread.abort_on_exception = true
         MatrixSdk.debug!
       end
 
-      bot = ChronicleBot.new(
+      ChronicleBot.new(
-        ENV["CHRONICLE_HOMESERVER"],
+        Chronicle::Config.matrix_homeserver,
-        ENV["CHRONICLE_ACCESS_TOKEN"]
+        Chronicle::Config.matrix_access_token
-      )
+      ).run
-
-      bot.run
     end
 
     # The bot
     class ChronicleBot
-      attr_reader :all_commands, :cmd_prefix
+      attr_reader :all_commands, :cmd_prefix, :scribe
 
       def initialize(hs_url, access_token)
         @hs_url = hs_url
@@ -65,6 +39,9 @@ module Chronicle
         @all_commands = {}
         @allowed_commands = {}
 
+        @scribe = Logger.new('chronicle.log')
+        @scribe.info('ChronicleBot') {'Initializing a new instance of Chronicle'}
+
         register_commands
         available_commands(self, %w[listcommands help])
       end
@@ -73,6 +50,7 @@ module Chronicle
       def available_commands(addon, commands)
         commands.each do |command|
           @all_commands[command] = addon
+          @scribe.info('ChronicleBot') {"Adding available command: #{command}"}
         end
       end
 
@@ -143,6 +121,10 @@ module Chronicle
         return unless msgstr =~ /^#{@cmd_prefix}#{cmds}\s*/
 
         msgstr.match(/^#{@cmd_prefix}(#{cmds})\s*/) do |m|
+          @scribe.info('ChronicleBot') {
+            "Running command: #{msgstr.split(' ')[0].strip}"
+          }
+
           @all_commands[m.to_s[1..-1].strip].matrix_command(message)
         end
       end