oss/direct-file
oss/direct-file
1
0
Fork 0
mirror of https://github.com/IRS-Public/direct-file.git synced 2025-06-27 20:25:52 +00:00
Code Wiki Activity

initial commit

Browse source
This commit is contained in:
sps-irs 2025-05-29 13:12:11 -04:00 • committed by Alexander Petros
parent 2f3ebd6693
commit e0d5c84451
3413 changed files with 794524 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

68
.gitignore vendored Normal file
View file

@ -0,0 +1,68 @@
*.class
maven-wrapper.jar
*.log
*.log.*
.DS_Store
audit_log.txt
# Local maven properties
**/application-local.*
# MS Word temporary files
~$*
# Jetbrains products
.idea
.idea_modules
/.worksheet/
*.iml
node_modules/
bin
# sbt specific
dist/*
target/
lib_managed/
src_managed/
project/boot/
project/plugins/project/
project/local-plugins.sbt
.history
.ensime
.ensime_cache/
.sbt-scripted/
local.sbt
# Bloop
.bsp
# Metals
.bloop/
.metals/
metals.sbt
.scala-build
src/main/resources/test/run/processed/
src/main/resources/test/run/tobatch/
src/main/resources/test/run/batched/
src/main/resources/test/run/submitted/
src/main/resources/test/run/input/
src/main/resources/test/run/acks/
src/main/resources/test/run/toprocess/
*~
\#*
#spotbugs
direct-file/**/src/main/resources/spotbugs/output/spotbugs.xml
direct-file/**/spotbugs/output/
# vscode
.vscode
!direct-file/.vscode
!direct-file/df-client/.vscode
direct-file/scripts/*.csv
# files created when building sbom.json files
sbom.json

21
LICENSE Normal file
View file

@ -0,0 +1,21 @@
# License
As a work of the [United States government](https://www.usa.gov/), this project is in the public domain within the United States of America.
Additionally, we waive copyright and related rights in the work worldwide through the CC0 1.0 Universal public domain dedication.
## CC0 1.0 Universal Summary
This is a human-readable summary of the [Legal Code (read the full text)](https://creativecommons.org/publicdomain/zero/1.0/legalcode).
### No Copyright
The person who associated a work with this deed has dedicated the work to the public domain by waiving all of their rights to the work worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
You can copy, modify, distribute, and perform the work, even for commercial purposes, all without asking permission.
### Other Information
In no way are the patent or trademark rights of any person affected by CC0, nor are the rights that other persons may have in the work or in how the work is used, such as publicity or privacy rights.
Unless expressly stated otherwise, the person who associated a work with this deed makes no warranties about the work, and disclaims liability for all uses of the work, to the fullest extent permitted by applicable law. When using or citing the work, you should not imply endorsement by the author or the affirmer.

451
ONBOARDING.md Normal file
View file

@ -0,0 +1,451 @@
# Onboarding
__Table of Contents__
1. [Quickstart](#quickstart)
1. [Codebase Overview](#codebase-overview)
2. [Local Environment Setup](#local-environment-setup)
1. [Software Installs](#software-installs)
* [Required Software](#required-software)
* [Optional Software](#optional-software)
* [Installing software using Homebrew](#installing-software-using-homebrew)
* [Installing software using SDKMAN!](#installing-software-using-sdkman)
2. [Source Code](#source-code)
3. [Environment Variables](#environment-variables)
4. [Building with Docker - RECOMMENDED](#building-with-docker)
5. [Building with command line — _local builds_)](#building-with-command-line)
* [Install shared dependencies](#install-shared-depedencies)
* [Stand up development containers](#stand-up-development-containers)
* [Build individual Spring Boot projects](#build-individual-spring-boot-projects)
* [Email-service](#email-service)
* [Build the client app](#build-the-client-app)
4. [Application Tests](#application-tests)
* [Code Coverage](#code-coverage)
# Quickstart
To run everything:
```bash
docker compose up -d --build
```
The backend application is available at http://localhost:8080 and the postgres database is exposed on port 5432 with username and password `postgres`.
When you're finished, tear it down with `docker compose down`.
We typically recommend running the front end components locally instead of through Docker to allow for hot reloading when making changes. Run the following from /direct-file/df-client
```bash
npm run start
```
The front end application is available at http://localhost:3000
# Codebase Overview
The below provides an introduction to various portions of the codebase. Most applications in our system come with a readme.md to explain what they are for. Follow the instructions there on how to build.
> n.b. Most, but not all, of the applications run in docker via running `docker-compose up --build` in the /direct-file directory. In particular, the applications that interact with MeF (status and submit) are read-only and are not included in the docker compose file.
>
> Direct File consists of a frontend React application, a suite of backend Java services, and a shared Scala library that ensures that taxpayers receive accurate error messages and UX flow for the tax rules that apply to them.
#### direct-file
Direct file is the home for the vast majority of our code. It is split into sub directories, many of which are applications in and of themselves.
#### direct-file/fact-graph-scala
The fact graph is the logical framework by which we store user information and calculate tax information. It is written in Scala and transpiled to JavaScript so that it can be used on the front end. It can be helpful to think of it like Excel. There are cells that a person writes in, and then there are a bunch of formulae that use the user entered information and calculated information.
#### direct-file/js-factgraph-scala
This is the module that contains the fact graph and the operations in the fact graph for the front end.
#### direct-file/backend
This application is the front door to our non-UI systems. It is responsible for integrating with an auth provider, generating tokens for our system, accepting user data, and maintaining user information. A bit more monolithic than we might have wanted but oh well.
#### direct-file/df-client/
Taxpayer facing frontend and UI. Utilizes the transpiled fact graph as the logical engine to control which screens are displayed. This is also the home for the flow, which is the configuration of which screens will be shown and under what conditions.
The frontend app is further in the `df-client-app` directory, whereas other frontend packages can exist at the `df-client` level. We use npm workspaces to connect our packages.
#### direct-file/submit
Submits tax returns to MeF
#### direct-file/status
Polls MeF for tax return acknowledgements
#### direct-file/email-service
SMTP relay service for sending emails to taxpayers, triggered on various system or MeF events
#### direct-file/state-api
Backend service responsible for the handling A2A traffic from state tax software providers via a REST API. These APIs are used to access federal return data (XML and return status). After a taxpayer submits their federal return, they may authorize the transfer of their federal return data with their state and the state tax software will pull that data through state-api.
## Local Environment Setup
__Table of Contents__
1. [Software Installs](#software-installs)
* [Required Software](#required-software)
* [Optional Software](#optional-software)
* [Installing software using Homebrew](#installing-software-using-homebrew)
* [Installing software using SDKMAN!](#installing-software-using-sdkman)
2. [Source Code](#source-code)
3. [Environment Variables](#environment-variables)
4. [Building with Docker - RECOMMENDED](#building-with-docker)
5. [Building with command line — _local builds_)](#building-with-command-line)
* [Install shared dependencies](#install-shared-depedencies)
* [Stand up development containers](#stand-up-development-containers)
* [Build individual Spring Boot projects](#build-individual-spring-boot-projects)
* [Email-service](#email-service)
* [Build the client app](#build-the-client-app)
### Software Installs
Table of Contents
1. [Required Software](#required-software)
2. [Optional Software](#optional-software)
3. [Installing software using Homebrew](#installing-software-using-homebrew)
4. [Installing software using SDKMAN!](#installing-software-using-sdkman)
#### Required Software
* Java
* Scala
* Maven
* SBT
* coursier
* Docker for Desktop
There are instructions below for using `Homebrew` or `SDKMAN` to install the required software. You should only follow one path or the other, unless the instructions tell you to do otherwise (i.e. `SDKMAN` doesn't currently support `coursier`, so you might use `Homebrew` for that).
#### Optional Software
* Homebrew
* SDKMAN!
* Visual Studio Code
* IntelliJ Community Edition
#### Installing software using Homebrew
You will need to install SBT (a build tool for Scala, does it mean Scala Build Tool?) in order to run some of the below steps, if using macOS, it is recommended that you install [Homebrew](https://brew.sh/) first and then use brew to install SBT
* Run the following command in a terminal
```sh
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
```
* Then either in your `.zshrc` or `.zprofile` file paste the following
```sh
# Set PATH, MANPATH, etc., for Homebrew.
eval "$(/opt/homebrew/bin/brew shellenv)"
```
* Then simply install SBT as documented [here](https://www.scala-sbt.org/1.x/docs/Installing-sbt-on-Mac.html#)
```sh
brew install sbt
```
* You will also need to install Scala, Coursier, Java, Maven, and Docker
```sh
brew install --cask docker
```
```sh
brew install scala maven openjdk@21 coursier
```
* If you had another JDK installed, you may need to link the java 21 JDK
```sh
brew unlink openjdk
```
```sh
brew link --force openjdk@21
```
* Add to your `.bash_profile` to ensure maven finds the correct version of java
```sh
export JAVA_HOME=$(brew --prefix openjdk@21)
```
* Configure Coursier to use the right version of Java for the direct file project. You may wish to add this to your .bash_profile or .zshrc to ensure it runs every time you load a new terminal.
```sh
eval "$(coursier java --jvm 21 --env)"
```
* Run Docker (from spotlight search on Mac). The Docker icon should appear in your status bar. You may wish to configure Docker to run at login/startup.
#### Installing software using SDKMAN!
Most of the project dependencies can be installed using [SDKMAN!](https://sdkman.io/), a CLI and API for managing SDKs from the JVM and beyond. SDKMAN! supports installation of Java, Scala, sbt, and Maven.
> Please note that support for installation of Coursier using SDKMAN! is currently under development, so this is the one tool we'll need to install manually.
* First, install SDKMAN! using the following command in a terminal:
```sh
curl -s "https://get.sdkman.io" | bash
```
* Then, open a new terminal OR run the following in the same shell to enable SDKMAN! in the current terminal:
```sh
source "$HOME/.sdkman/bin/sdkman-init.sh"
```
> SDKMAN! will configure your $JAVA_HOME automatically to point to `"$HOME/.sdkman/candidates/java/current"` by default.
* You can install the latest stable version of your SDK tools using its canonical name without specifying a version:
> You can use the `sdk list {package}` command to list out available versions. (eg., `sdk list java` will show you available OpenJDK builds).
```sh
sdk install java
```
```sh
sdk install sbt
```
```sh
sdk install scala
```
```sh
sdk install maven
```
* Until SDKMAN! supports Coursier officially, you'll need to manually install it here.
* Either consult the Homebrew instructions above, or follow their official [CLI installation](https://get-coursier.io/docs/cli-installation) steps:
* On Apple Silicon (M1, M2, ...):
```sh
$ curl -fL https://github.com/VirtusLab/coursier-m1/releases/latest/download/cs-aarch64-apple-darwin.gz | gzip -d > "$HOME/.local/bin/cs"
```
* Otherwise:
```sh
curl -fL https://github.com/coursier/launchers/raw/master/cs-x86_64-apple-darwin.gz | gzip -d > "$HOME/.local/bin/cs"
```
* Then
```sh
chmod +x cs
```
```sh
./cs setup
```
You'll want to make sure that `cs` is available on your `$PATH`.
* Then, configure Coursier to use the right version of Java for the direct file project. You may wish to add this to your .bash_profile or .zshrc to ensure it runs every time you load a new terminal.
```sh
eval "$(coursier java --jvm 21 --env)"
```
* Run Docker (from spotlight search on Mac). The Docker icon should appear in your status bar. You may wish to configure Docker to run at login/startup.
### Source Code
* Clone this repo
### Environment Variables
1. Add the following environment variables to your system, on macOS you can add the following lines to your shell's root config file (i.e. the `.zshenv`, `.zshrc`, or `.bashrc` file). Note that you will need to edit most variables.
```sh
export MEF_REPO=~
export INSTALL_MEF=0
export LOCAL_WRAPPING_KEY="9mteZFY+gIVfMFywgvpLpyVl+8UIcNoIWpGaHX4jDFU="
export MEF_SOFTWARE_ID="[mef-software-id]"
export MEF_SOFTWARE_VERSION_NUM="2023.0.1"
export STATUS_ASID="[status-asid]"
export STATUS_EFIN="[status-efin]"
export STATUS_ETIN="[status-etin]"
export SUBMIT_ASID=$STATUS_ASID
export SUBMIT_EFIN=$STATUS_EFIN
export SUBMIT_ETIN=$STATUS_ETIN
export DF_TIN_VALIDATION_ENABLED=false
export DF_EMAIL_VALIDATION_ENABLED=false
export STATUS_KEYSTOREALIAS="[keystore-alias]"
export STATUS_KEYSTOREBASE64="[base64-encoded-keystore]"
export STATUS_KEYSTOREPASSWORD="[keystore-password]"
export SUBMIT_KEYSTORE_KEYSTOREALIAS=$STATUS_KEYSTOREALIAS
export SUBMIT_KEYSTORE_KEYSTOREBASE64=$STATUS_KEYSTOREBASE64
export SUBMIT_KEYSTORE_KEYSTOREPASSWORD=$STATUS_KEYSTOREPASSWORD
export SUBMIT_ID_VAR_CHARS="zz"
export GIT_COMMIT_HASH="$(cd /path/to/direct-file && git rev-parse --short main)"
```
2. From the root directory of this repo, run the following command to generate a value for LOCAL_WRAPPING_KEY:
```sh
./direct-file/scripts/local-setup.sh
```
3. Re-load your environment so that the new `LOCAL_WRAPPING_KEY` value is loaded. If you set the values in one of your shell dotfiles (e.g. `.zshrc`), open a new terminal.
### Building with Docker
1. To work with the Direct File docker setup, change into the `direct-file` subdirectory of this repo.
```sh
cd direct-file/
```
#### Default Services/Containers
1. Run the following command to build and start the default services and containers:
```sh
docker compose up -d --build
```
1. You should see the following (among other) containers start up:
* direct-file-app — df-client | `df-client`
* direct-file-db
* state-api-db
* direct-file-csp-simulator — csp-simulator | `/utils/csp-simulator`
* localstack
* direct-file-api — api | `/backend`
* state-api — state-api | `/state-api`
* direct-file-email-service — email-service | `/email-service`
* redis
##### Troubleshooting
1. If you get a build error with the `docker compose` command, you can try a few things.
1. If the error is related to running out of memory, you may need to increase the amount of memory you've allocated to docker to 16 GB.
2. Otherwise, you can try building without cache:
```sh
docker compose build --no-cache
```
and then re-run the previous command:
```sh
docker compose up -d
```
#### Resources
That's it!
Some quick links:
* API documentation for the backend app can be viewed at http://localhost:8080/df/file/api/swagger-ui/index.html
* To access Direct File through the CSP simulator in browser, go to http://localhost:5000/ and use any email and select `IAL2` to login
### Building with command line
1. [Install shared dependencies](#install-shared-dependencies)
2. [Stand up development containers](#stand-up-development-containers)
3. [Build individual projects](#build-individual-spring-boot-projects)
#### Install shared dependencies
*Note: Direct File shell scripts use Maven Wrapper; therefore they need to be executed from a working directory where it is present*
1. Navigate to the `direct-file/libs` directory which has the Maven Wrapper.
```sh
cd direct-file/libs
```
2. Run the `build-dependencies.sh` to build and install Direct File shared dependencies.
```sh
INSTALL_MEF=1 ../scripts/build-dependencies.sh
```
#### Stand up development containers
Use Docker to build database containers and AWS mock services (referred to as "localstack")
```sh
docker compose up -d db mef-apps-db localstack
```
The command below will display all running containers and can be used to validate the above command was successful
```shell
docker ps
```
If successful, you should see three images running: localstack, direct-file-mef-apps-db, and direct-file-db.
#### Build individual Spring Boot projects
__Spring Boot projects__
* backend
* email-services
* state-api
* status
* submit
Navigate to a `<project>` directory and use the Spring Boot Maven plugin to build and run.
```sh
./mvnw spring-boot:run -Dspring-boot.run.profiles=development
```
##### Email-service
```sh
# make sure the docker container for state-api is down as the following commands use the same localhost port
docker compose down state-api
# will start up the application using the blackhole profile
./mvnw spring-boot:run -Dspring-boot.run.profiles=development
# prints a log message to the console instead of attempting to send an email
./mvnw spring-boot:run -Dspring-boot.run.profiles=blackhole
# will attempt
./mvnw spring-boot:run -Dspring-boot.run.profiles=send-email
```
#### Build the client app
Need to run/develop the client app? Check out the [df-client/README](/direct-file/df-client/README.md) for info on getting your local environment setup.
## Application Tests
Each application has its own set of tests. To run server-side tests within an app, navigate to the root of the app. Run:
```sh
cd direct-file/<project>
./mvnw test
```
To run a test individually, run `./mvnw -Dtest=<Name of Test> test` with the test name. For example:
```sh
./mvnw -Dtest=TaxReturnServiceTest test
```
__NOTE__ - add the `-X` flag to any maven command to switch on debug logging
```sh
./mvnw spring-boot:run -Dspring-boot.run.profiles=development -X
./mvnw -Dtest=TaxReturnServiceTest test -X
```
### Code coverage
We use a plugin called [Jacoco Maven](https://www.eclemma.org/jacoco/trunk/doc/maven.html) to run code coverage.
To run code coverage in any particular app:
```sh
./mvnw jacoco:report
```
To view the generated report, go to `<app_name>/target/site/jacoco/index.html` and open it in a browser.

33
README.md Normal file
View file

@ -0,0 +1,33 @@
# Direct File
[Direct File](https://directfile.irs.gov) is a service from the United States Government that provides taxpayers the option to electronically file their federal tax return for free, directly with the Internal Revenue Service (IRS). Direct File is an interview-based service that is intended to work as well on a mobile phone as it does on a laptop, tablet, or desktop computer. It is available in English and Spanish and is designed to be accessible to taxpayers who have a variety of attitudes, aptitudes, abilities, and access needs.
Direct File interprets the United States' [Internal Revenue Code (26 USC)](https://www.irs.gov/privacy-disclosure/tax-code-regulations-and-official-guidance) as plain language questions, the answers to which should be known to taxpayers without need of external instructions or publications. Taxpayers' answers are then translated into standard tax forms and transmitted to the IRS's [Modernized e-File (MeF)](https://www.irs.gov/e-file-providers/modernized-e-file-program-information) API, which is available for authorized public use. These questions and logic, developed in close collaboration with the IRS [Office of Chief Counsel](https://www.irs.gov/about-irs/office-of-chief-counsel-at-a-glance), as well as the associated test cases and scenarios, may be useful for others working on products that need to accurately interpret United States tax law as of Tax Year 2024.
Direct File also incorporates the Fact Graph, a declarative, XML-based knowledge graph data structure that is designed to reason about incomplete information, such as a partially completed tax return. The Fact Graph is written in the Scala programming language; it runs on the JVM on the backend and is transpiled via [Scala.js](https://www.scala-js.org) to run on the client as well. Direct File's Fact Graph is not domain-specific, and it may be useful to revenue agencies and as a reference for business rules engine implementations.
Although Direct File only files federal tax returns, United States taxpayers also have state and local filing obligations. Direct File facilitates the completion of these obligations by enabling taxpayers to optionally import their federal return data into a third-party tool that can file state and/or local taxes, without needing to reenter information. This transaction is enabled via a State API, which transfers both standard MeF XML as well as an enriched JSON format that includes additional data elements that were identified as being useful to state revenue agencies to streamline the state tax experience.
Direct File was developed by an in-house team of technologists at the IRS. The blended, cross-agency team included support from [USDS](https://www.usds.gov) and [GSA](https://www.gsa.gov/), as well as vendor teams [TrussWorks](https://truss.works), [Coforma](https://coforma.io), and [ATI](https://atisolutions.us/).
For a more details on the program and its history see https://www.irs.gov/pub/irs-pdf/p5969.pdf and https://www.irs.gov/filing/irs-direct-file-for-free
## Where do I start?
See [ONBOARDING.md](/ONBOARDING.md) if you want to jump into running Direct File locally
## Exempted Code
Not all source code, documentation and metadata used in the development of Direct File is included in this repository. Specifically, any code or data that is considered Personally Identifiable Information (PII), Federal Tax Information (FTI),
Sensitive But Unclassified (SBU), or source code developed for National Security Systems (NSS), as defined in 40 U.S.C. § 11103, is exempt. Due to these restrictions, certain pieces of functionality have been removed or rewritten.
# Authorities
Legal foundations for work include:
* Source code Harmonization And Reuse in Information Technology Act" of 2024, Public Law 118 - 187
* OMB Memorandum M-16-21, “Federal Source Code Policy: Achieving Efficiency,
Transparency, and Innovation through Reusable and Open Source Software,” August 8,
2016
* Federal Acquisition Regulation (FAR) Part 27 – Patents, Data, and Copyrights
* Digital Government Strategy: “Digital Government: Building a 21st Century Platform to
Better Serve the American People,” May 23, 2012
* Federal Information Technology Acquisition Reform Act (FITARA), December 2014
(National Defense Authorization Act for Fiscal Year 2015, Title VIII, Subtitle D)
* E-Government Act of 2002, Public Law 107-347
* Clinger-Cohen Act of 1996, Public Law 104-106

1
direct-file

@ -1 +0,0 @@
Subproject commit 6f05405ac4c8d638c5f3d840211143585a153cc2

10
direct-file/.vscode/settings.json vendored Normal file
View file

@ -0,0 +1,10 @@
{
"xml.format.maxLineWidth": 120,
"[xml]": {
"editor.defaultFormatter": "redhat.vscode-xml",
"editor.tabSize": 2
},
"files.watcherExclude": {
"**/target": true
},
}

172
direct-file/README.md Normal file
View file

@ -0,0 +1,172 @@
# direct-file
If you're ready to set up your local developer environment, go directly to [ONBOARDING.md](/ONBOARDING.md) and return back here for background information.
## Docker
First, some things that must be true for this to work:
* You have cloned this repository.
* You don't have other services occupying ports 3000, 8080, or 5432 (or have set alternate ports with environment variables as described below)
### Additional configuration for Apple M2 Laptop
If you are running docker on an Apple M2 laptop, you may also need to change the default file sharing implementation in your docker settings.
If you see the error `dependency failed to start: container direct-file-db is unhealthy` when attempting to start up the docker instance, try the following steps.
From the docker desktop:
* Enter the settings menu (click the gear icon in the top right)
* On the **General** tab, for `Choose file sharing implementation for your containers`
* Select the `gRPC FUSE` option ,
* Click the `Apply & Restart` button on the bottom right.
## Important configuration variables
| Name | Required | Default | Description |
|-------------------------|----------|----------------|-----------------------------------------------------------------------------------------------------------------------------------------|
| `DF_DB_USER_ID` | No | 999 | User id used to run the database (if you set this, it likely will be to the value of `id -u`. |
| `DF_DB_GROUP_ID` | No | 999 | Group id used to run the database (if you set this, it likely will be to the value of `id -g`. |
| `DF_DB_PORT` | No | 5432 | Port the backend api database will be exposed on outside of the docker network. |
| `MEF_APPS_DB_PORT` | No | 32768 | Port the submit/status database will be exposed on outside of the docker network |
| `STATEAPI_DB_PORT` | No | 5433 | Port the state api database will be exposed on outside of the docker network |
| `DF_CSPSIM_PORT` | No | 5000 | Port the CSP Simulator will be exposed on outside of the docker network. |
| `DF_EXTSVCSIM_PORT` | No | 5001 | Port the External Service Simulator (ESSAR, etc) will be exposed on outside of the docker network. |
| `DF_API_PORT` | No | 8080 | Port the backend app is exposed to outside of docker. |
| `DF_STATUS_PORT` | No | 8082 | Port the status app is exposed to outside of docker. |
| `DF_SUBMIT_PORT` | No | 8083 | Port the submit app will run on and is exposed on outside of docker. |
| `DF_FE_PORT` | No | 3000 | Port that will be exposed to access the frontend through docker. This currently does not work for plain `npm start` outside of docker. |
| `DF_SCREENER_PORT` | No | 3500 | Port to access screener in docker. |
| `DF_PROMETHEUS_PORT` | No | 9090 | Port that will be exposed to access the Prometheus dashboard from docker |
| `DF_GRAFANA_PORT` | No | 3030 | Port that will be exposed to access the Grafana dashboard from docker |
| `DF_CLIENT_PUBLIC_PATH` | No | `/df/file` | Path prefix the client will be served from publicly. This is embedded into build process and applies to docker builds. |
| `DF_API_PUBLIC_PATH` | No | `/df/file/api` | Path prefix the api will be served from publicly. |
| `MAVEN_OPTS` | No | | Extra options to pass to maven, especially useful for setting a proxy. |
| `DF_LISTEN_ADDRESS` | No | 127.0.0.1 | Listen address for docker services. Set to "0.0.0.0" to listen on everything. |
| `DF_DISABLE_AUTO_LOGOUT` | No | false | Disable autologout. This env var is only read by the node app through `VITE`, and only on `development`.
### Build
To build the factgraph, api, frontend, and setup a database simply run:
```bash
docker compose build
```
Then, to start it all:
```bash
docker compose up -d
```
Now you can use the application with a browser at http://localhost:5000 (or the port specified as `DF_CSPSIM_PORT`). You will be accessing the authentication simulator directly, which will pass your traffic on to the client and backend api services in docker.
The backend api is at http://localhost:8080 (or `DF_API_PORT`) and the database is exposed on port 5432 (or `DF_DB_PORT`).
#### Common configurations
The default configuration if you run `docker compose up` will let you access the application in the browser through the authentication simulator at `DF_CSPSIM_PORT`.
Paths prefixed with `DF_CLIENT_PUBLIC_PATH` will be passed to the client and those prefixed with `DF_API_PUBLIC_PATH` will be passed to the API. The most specific path match will be used, so these public prefixes may be nested.
Although the client and API services are behind authentication, the default configuration exposes their ports externally. You can load the client in a browser directly, but API requests will fail because those requests would (due to client configuration) be through the CSP simulator and the browser would not have a valid cookie.
![Image of default docker compose development configuration](../docs/images/dev_config_docker_compose.png)
For client development, you can bypass the authentication simulator. First, make sure the docker container for the client is not running (`docker compose rm -sf df-client`). Next, start the client with `npm start` or `docker_dev_server.sh`. Once it is started, access the client directly in the browser at `DF_CLIENT_PORT`.
![Image of common client development configuration](../docs/images/dev_config_client.png)
Other local configurations are possible.
### Monitoring
The applications use [OpenTelemetry](https://opentelemetry.io/docs/) locally for instrumenting observability metrics.
To enable and run the monitoring functionality locally, run:
```base
JAVA_TOOL_OPTIONS="-javaagent:/opentelemetry-javaagent.jar" docker compose --profile monitoring up -d --build
```
You can view what metrics we currently track through the Prometheus dashboard via http://localhost:9090 by default or `http://localhost:{DF_PROMETHEUS_PORT}` if `DF_PROMETHEUS_PORT` was set.
You can access and define dashboards through Grafana via http://localhost:3030 by default, or `http://localhost:{DF_GRAFANA_PORT}` if you've overridden the port. The default username is `admin` and the default password is `directfile`.
### Removing a service from docker compose
When you have a service running that you want to stop/remove, use:
```bash
docker compose rm --stop --force service-name
```
It can then be re-created and started with:
```bash
docker compose up -d service-name
```
### Git hooks
The maven Spotless plugin and the frontend `prettier` hook is used to help with standardizing formatting. To check backend formatting of your current changes, run `./mvnw spotless:check`, and to apply those changes, use `./mvnw spotless:apply`.
#### Pre-commit
To make it easier to use, a `pre-commit` configuration has been added at the root of the repository. You can install it with:
```bash
# linux
apt install pre-commit
# macos
brew install pre-commit
# and then, from a shell with cwd inside this repo:
pre-commit install
pre-commit install --hook-type pre-push
```
If you want to disable the checks, you can use:
```bash
pre-commit uninstall
```
or run your git commit with a `--no-verify` flag.
### Enable Optional Monitoring Service
Starts an optional OpenTelemetry collector, Prometheus, and Grafana instance for testing purposes.
```bash
JAVA_TOOL_OPTIONS="-javaagent:/opentelemetry-javaagent.jar" docker compose --profile=monitoring up -d --build
```
### Enable Debug of Containerized App
```bash
docker compose -f docker-compose.yaml -f docker-compose.debug.yaml up -d
```
In Visual Studio Code, add following to launch.json:
```
{
"type": "java",
"name": "Attach to Remote Program",
"request": "attach",
"hostName": "localhost",
"port": "5005",
"projectName": "directfile-api"
}
```
This will enable debugging for the backend project, and the same approach can be applied to other projects.
### Setup Redrive Policy for DLQ using CLI
Some CLI commands for reference to set redrive policy locally.
```
awslocal sqs list-queues (all DLQs prefixed by dlq-)
awslocal sqs get-queue-attributes --queue-url <dlq-queue-url> --attribute-names QueueArn (get ARN for DQL)
awslocal sqs set-queue-attributes --queue-url <queue-url> --attributes '{"RedrivePolicy":"{\"deadLetterTargetArn\":\"<dlq-queue-arn>\",\"maxReceiveCount\":\"2\"}"}'
awslocal sqs send-message --queue-url <source-queue-url> --message-body "Your message content"
awslocal sqs receive-message --queue-url <source-queue-url> --message-attribute-names All
```

5
direct-file/backend/.dockerignore Normal file
View file

@ -0,0 +1,5 @@
**/application-local.*
.env*
.git/
Dockerfile*
**/target/

6
direct-file/backend/.gitattributes vendored Normal file
View file

@ -0,0 +1,6 @@
# Linux start script should use lf
/gradlew text eol=lf
# Windows script files should use crlf
*.bat text eol=crlf

48
direct-file/backend/.gitignore vendored Normal file
View file

@ -0,0 +1,48 @@
HELP.md
/src/main/resources/application-local.yaml
target/
!**/src/main/**/target/
!**/src/test/**/target/
!**/src/main/resources/certs/
*.jar
/src/main/resources/spotbugs/output/spotbugs.xml
src/main/java/META-INF
### STS ###
.apt_generated
.classpath
.factorypath
.project
.settings
.springBeans
.sts4-cache
### IntelliJ IDEA ###
.idea
*.iws
*.iml
*.ipr
### NetBeans ###
/nbproject/private/
/nbbuild/
/dist/
/nbdist/
/.nb-gradle/
build/
!**/src/main/**/build/
!**/src/test/**/build/
### VS Code ###
.vscode/
### docker-related ###
/docker/db/postgres/data
### Gradle ###
.gradle/
end-to-end-result*
### XML Related ###
/src/main/java/gov/irs/directfile/api/xmlgeneration/xml

5
direct-file/backend/.liquibase.properties Normal file
View file

@ -0,0 +1,5 @@
changeLogFile=db/changelog.yaml
url=jdbc:postgresql://localhost:5432/directfile
username=postgres
password=postgres
changesetAuthor=directfile

19
direct-file/backend/.mvn/wrapper/maven-wrapper.properties vendored Normal file
View file

@ -0,0 +1,19 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
wrapperVersion=3.3.2
distributionType=only-script
distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.9.7/apache-maven-3.9.7-bin.zip

54
direct-file/backend/Dockerfile-local Normal file
View file

@ -0,0 +1,54 @@
#syntax=docker/dockerfile:1.7-labs
# Note: This uses some relatively new features, so make sure docker is
# up-to-date on your system or this build may fail
# 1. build factgraph
# For this step to work, you must define `factgraph-repo` in either your
# docker-compose.yaml's `additional_contexts` or from the command line with
# `docker buildx build --build-context factgraph-repo=/some/path [other args]`
FROM sbtscala/scala-sbt:eclipse-temurin-alpine-21.0.2_13_1.9.9_3.3.3 AS factgraph-builder
WORKDIR /build/
COPY --from=factgraph-repo js/src/ js/src/
COPY --from=factgraph-repo jvm/src/ jvm/src/
COPY --from=factgraph-repo project/build.properties project/plugins.sbt project/
COPY --from=factgraph-repo shared/ shared/
COPY --from=factgraph-repo build.sbt .
RUN sbt compile package publishM2
# 2. build shared dependencies
FROM eclipse-temurin:21-jdk-alpine AS shared-dependencies-builder
COPY --from=factgraph-builder /root/.m2/repository/gov/irs/factgraph/fact-graph_3/ /root/.m2/repository/gov/irs/factgraph/fact-graph_3/
ARG MAVEN_OPTS=""
WORKDIR /build/
COPY --from=config . ./config/
COPY --from=boms . ./boms/
WORKDIR /build/libs/
COPY --from=shared-libs .mvn/wrapper/maven-wrapper.properties .mvn/wrapper/
COPY --from=shared-libs mvnw ./
COPY --from=shared-libs --parents **/pom.xml ./
RUN ./mvnw dependency:resolve -P resolve
COPY --from=shared-libs starters/ ./starters/
COPY --from=shared-libs data-models/ ./data-models/
RUN ./mvnw install
# 5. build backend spring api
FROM shared-dependencies-builder AS api-builder
ARG MAVEN_OPTS=""
ENV MEF_REPO /mef-repo
COPY --from=config . /config/
WORKDIR /build/
COPY mvnw ./
COPY .mvn/wrapper/maven-wrapper.properties .mvn/wrapper/
COPY pom.xml ./
RUN ./mvnw dependency:resolve
# copy source tree after dependency resolution so source changes don't force re-download of all dependencies
COPY src/ /build/src/
RUN ./mvnw package
# 6. bundle backend api jar into runnable image
FROM eclipse-temurin:21-jre-alpine
ENV SPRING_PROFILES_ACTIVE=development
COPY --from=api-builder /build/target/directfile-api-0.0.1-SNAPSHOT.jar /app.jar
RUN adduser --system --no-create-home jar-runner
USER jar-runner
CMD ["java", "-jar", "/app.jar"]

365
direct-file/backend/README.md Normal file
View file

@ -0,0 +1,365 @@
## Key Concepts
## Identifiers
### Tax Return ID
The UUID identifier (e.g. 4638655a-5798-4174-a5a0-37cc3b3cd9a0) that identifies the entire experience a taxpayer has with Direct File for one filing season independent of submissions. We generate this ID once at return creation time. MeF has no knowledge of this ID
Example: My first submission is rejected by MeF and my second is accepted. The tax return ID will be the same for both
### MeF Submission ID
The string identifier (e.g. 55555620230215000001) that identifies each submission within MeF. We generate this ID at each return's submission time.
Example: My first submission is rejected by MeF with submission ID 55555620230215000001 and my second submission is accepted with submission ID 54444420240215000004. The tax return ID will be the same for both submissions
### Receipt ID
The UUID identifier that identifies receipt of the submission by MeF. MeF generates this ID for each submission we send it. If no receipt ID exists for a submission, then MeF didn't receive it.
Example: My first submission is rejected by MeF with receipt ID 2d59a07d-57ef-4392-8196-48ac29dce023 and my second submission is accepted with receipt ID 0ac15058-9352-49f8-9b84-5e3faed41676.
Example: MeF is accepting submissions but isn't processing them. My first submission is submitted to MeF and enqueued to the backlog of submissions to process, and MeF returns receipt ID 2d59a07d-57ef-4392-8196-48ac29dce023. I do not receive an acknowledgement (see below) until MeF is back online which tells me if my return is accepted or rejected
### Acknowledgement
The term used for a processed submission in MeF that has a status associated with it (accepted or rejected). Associates to a submission Id and receipt Id.
Example: My first submission is submitted and acknowledged by MeF with a rejected status, with receipt ID 2d59a07d-57ef-4392-8196-48ac29dce023 . My second submission is submitted and acknowledged by MeF with an accepted status, with receipt ID 0ac15058-9352-49f8-9b84-5e3faed41676.
## Tax Logic
This is a very brief introduction to writing tax logic and the fact graph. The /docs and /direct-file/df-client repos go much farther in depth on these topics and are worth reading!
### Introduction
Direct File's core data model for taxes is a graph, which we call the 'fact graph'. The rationale behind using graph-structures for modeling tax calculations is best articulated in https://arxiv.org/pdf/2009.06103.
### Reasoning about the fact graph
The fact graph is a huge collection of facts, both collected from the user (`writable`) and then `derived` from the `writable` and other `derived` facts. The fact graph is namespaced with a default private scope for each fact unless the need to be exported
**Writable facts**: Facts that are populated by user entered data.
**Derived Facts**: Facts that are calculated based on other facts.
Writing functional tax logic spans both front- and back-end. The main areas where this code lives is:
* [./direct-file/backend/src/main/resources/tax](./direct-file/backend/src/main/resources/tax) for facts and flow additions
* [./direct-file/df-client/df-client-app/src/flow](./direct-file/df-client/df-client-app/src/flow) to add pages to the flow
* [./direct-file/df-client/df-client-app/src/locales/](./direct-file/df-client/df-client-app/src/locales) for content
Tests are different for each type of work, but are primarily written in [./direct-file/df-client/df-client-app/src/test](./direct-file/df-client/df-client-app/src/test).
### Knockouts
Writing or editing knockouts requires you to create facts to prove or disprove something about a taxpayer's situation to knock them out because Direct File doesn't support their situation.
This includes:
* Figuring out the criteria for a knockout. This is often in the ticket and I encourage you to ask all teh questions
* Creating or using existing facts that support a knockout case
* If you're adding a knockout to the flow, adding that in
* Creating a knockout stub in the correct spot in the flow
* Tests test tests. Knockouts are mostly tested with the functional flow modality
### API Documentation
When running the backend application locally, OpenApi Documentation can be viewed at:
http://localhost:8080/df/file/api/swagger-ui/index.html
If you want to use the endpoints and be associated with a specific user, you should use the uuid
in the `external_id` column of the `users` table in the direct file db:
```sql
select
users.id,
users.external_id
from
users
left join taxreturn_owners on
users.id = taxreturn_owners.owner_id
left join taxreturns on
taxreturn_owners.taxreturn_id = taxreturns.id
where
taxreturns.id = '{taxReturnId}';
```
### Local
Backend relies on locally installed Maven packages in order to build; therefore, you can run the `/scripts/build-project.sh` which will install the shared dependencies
```sh
INSTALL_MEF=0 ../scripts/build-project.sh
```
#### Spring Boot Maven plugin
[Spring Boot](https://docs.spring.io/spring-boot/)
&nbsp;&nbsp;&nbsp;&nbsp;
&rarr; [Build Tools Plugins](https://docs.spring.io/spring-boot/build-tool-plugin/index.html)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&rarr; [Maven Plugin](https://docs.spring.io/spring-boot/maven-plugin/index.html)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&rarr; [Running your Application with Maven](https://docs.spring.io/spring-boot/maven-plugin/run.html)
Executing the Spring Boot Maven plugin `run` command will compile, verify and then run the application. Default profiles have been defined as configuration within the `pom.xml`
```sh
./mvnw spring-boot:run
```
A "debug" profile has been defined that enables all the HTTP actuator endpoints and to always show values as configured in `application-debug.yaml` The logging format is restored to the Spring Boot defaults with the "debug" profile to make log entries easier to read vs the current customization to format as JSON for use with log aggregators such as Splunk.
##### Configuration related endpoints
* [Conditions Report](http://localhost:8080/df/file/api/actuator/conditions) &mdash; `/actuator/conditions`
* [Configuration Properties](http://localhost:8080/df/file/api/actuator/configprops) &mdash; `/actuator/configprops`
* [Environment Variables](http://localhost:8080/df/file/api/actuator/env) &mdash; `/actuator/env`
## Development
### Database
The backend relies on a postgres database that can be started independent of other services (from this directory):
If you want to expose the database on a different port than the default (5432), set the environment variable `DF_DB_PORT` to the port you prefer.
```bash
docker compose up -d db
```
This database can be used by itself for local development while running the Spring application on the CLI, through your IDE, or in a container.
### Localstack
The backend uses an AWS mock library `localstack` to enable offline development. We use it specifically for artifact storage in S3 and SQS messaging queues. localstack runs in a docker container which can be started by running:
```bash
docker compose up -d localstack
```
For troubleshooting you can open a shell in the localstack container (```docker exec -it localstack sh```) and run commands AWS CLI commands style commands -- just replace aws with awslocal. Try `awslocal -h` for more details.
### Application
Requirements:
* Java 21 JDK
* [SBT](https://www.scala-sbt.org/) (1.9)
#### Initial Setup
See [ONBOARDING.md - Local Environment Setup](../../ONBOARDING.md#local-environment-setup) for details on getting set up.
Note that if you would like to develop the backend locally and outside of docker, follow the
[ONBOARDING.md - Building with command line](../../ONBOARDING.md#building-with-command-line) instructions.
##### Ports
By default, the application will run on port 8080. If you want to change this, set the environment variable
`DF_API_PORT`. If you have also changed the frontend port, make sure `DF_FE_PORT` has been exported prior to running the
application to ensure fake login redirects work correctly.
#### Remote debugging
To set up remote debug:
```
./mvnw spring-boot:run -Dspring-boot.run.profiles=development -Dspring-boot.run.jvmArguments="-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005"
```
#### Running in docker
The application can be run in a container locally, and it might be useful, but probably isn't the most convenient way to
do development right now. If you want to use it, cycles essentially are:
```bash
# make some changes, then:
docker compose up -d api
```
Recommended usage at this time is use an IDE for backend development and run the database and frontend in docker.
### Mock Data Import Service
The mock Data Import Service is configured to run in the following environments where the _mock_ Spring Profile has been activated
* local development
* local Docker
#### Reference profiles
The following reference profiles
* `/src/main/resources/dataimportservice/mocks/marge.json`
* `/src/main/resources/dataimportservice/mocks/homer.json`
##### VS Code REST Client
`/src/test/resources/endpoint.http`
#### How to add additional profiles?
To add additional profiles to the mock service, add JSON files to `/src/main/resources/dataimportservice/mocks/` where the file name matches the profile value you will be passing in the `x-data-import-profile` request header
## Running tests
Run the test suite that is executed in CI with:
```bash
./mvnw test
```
## Database migrations
The database schema is managed by [liquibase](https://www.liquibase.org).
Migration history and state are stored in tables named `databasechangelog` and `databasechangeloglock`. The migrations themselves are stored in [changelog.yaml](src/main/resources/db/changelog.yaml) and accompanying changesets in the `migrations` directory (relative to the changelog).
Liquibase commands can be run using the maven plugin, which is configured by default in this project to run against the local development database. This configuration is in [.liquibase.properties](.liquibase.properties).
To override this configuration and connect to a different database, use command line options. For example:
```
./mvnw liquibase:generateChangeLog \
-Dliquibase.changeLogFile=db/changelog.yaml \
-Dliquibase.url=jdbc:postgresql://localhost:5432/directfile \
-Dliquibase.username=postgres \
-Dliquibase.password=postgres \
```
Substitute the liquibase command you want to run (run `./mvnw liquibase:help` for a list of commands and more information)
### Applying migrations
When the application starts, all migrations that have not been applied will be applied. This behavior is similar to how `ddl-auto` as long as you aren't making the schema changes yourself.
### Creating new migrations
To add a new migration, add a new migration to [the migrations folder](src/main/resources/db/migrations) and add new `changeSet`(s).
When doing this, it is a good idea to also provide a `rollback` block to reverse each `changeSet`.
A working `rollback` block is also very useful when iterating on a migration locally.
Liquibase can infer how to roll back certain changes automatically. For more information on which commands should have
custom rollback statements, see [Automatic and Custom Rollbacks](https://docs.liquibase.com/workflows/liquibase-community/automatic-custom-rollbacks.html).
An example `changeSet` for adding a column is:
```
- changeSet:
id: sample-changeset
author: directfile
comment: sample changeset adding a column
changes:
- addColumn:
tableName: users
columns:
- column:
name: something_new
type: TEXT
remarks: this is a new column
rollback:
- dropColumn:
tableName: users
columns:
- column:
name: something_new
```
### Rolling back migrations
To roll back the most recent migration, you could:
```
./mvnw liquibase:rollback -Dliquibase.rollbackCount=1
```
### Static Analysis: Spot Bugs and PMD
We use [SpotBugs](https://spotbugs.readthedocs.io/en/stable/bugDescriptions.html) and [PMD](https://pmd.github.io/pmd/index.html) for static code analysis in this app. The app is configured to have pre-commit hooks run SpotBugs and PMD.
Spot Bugs is a static analysis tool for java projects. SpotBugs runs against _compiled_ code.
Be sure to run `./mvnw compile` to ensure that SpotBugs runs against the latest version of your code.
PMD is a static analysis tool that runs against the source code of the project. You can
run `./mvnw pmd:check` to check for PMD violations or `./mvnw pmd:pmd` to generate the pmd report.
Spotbugs and PMD both generate static analysis reports that can be used to resolve issues in the project.
PMD is configured via [xml file](src/main/resources/pmd/static-analysis-ruleset.xml) that specifies the linting rules we adhere to.
**How do I see the reports?**
To see a formatted HTML page for the static analysis reports you can run:
```bash
./mvnw clean compile site:run
```
This will start a site at `localhost:8080`. Navigate to `Project Reports` and then click on PMD or Spotbugs to view errors in the app.
If you want to ignore the pre-commit hook that runs static analysis, do:
`git commit --no-verify`
To generate each XML report, you can run:
```bash
./mvnw compile spotbugs:spotbugs
```
```bash
./mvnw pmd:pmd
```
To check if the project currently passes static analysis:
```bash
./mvnw compile spotbugs:check
```
```bash
./mvnw pmd:check
```
SpotBugs also offers a local gui that displays information based on the output of spotbugs. Calling compile before spotless:gui, ensures
we have all the latest changes reflected in the spotbugs report.
```bash
./mvnw clean compile spotbugs:gui
```
I've also configured the build to generate the spotbugs report when you run `./mvnw clean compile`. SpotBugs looks at the generated target folder
of the project, so doing a `./mvnw clean compile` will ensure you're seeing the latest spotbugs report.
**How do I resolve my SpotBugs / PMD Errors?**
Spotbugs and PMD provide references for how to fix warnings.
These are the rules for Spotbugs, you can search the page for the warning to understand how to fix it:
- Find Security Bugs Reference: https://find-sec-bugs.github.io/bugs.htm
- SpotBugs Reference: https://spotbugs.readthedocs.io/en/latest/bugDescriptions.html
PMD also has a page with rules and how to address them:
PMD Rule Reference: https://docs.pmd-code.org/latest/pmd_rules_java.html
**For more information on Spotbugs:**
- Docs Site: https://spotbugs.readthedocs.io/en/latest/introduction.html
- Github Repo: https://github.com/spotbugs/spotbugs
- Maven Spotbugs Plugin Docs: https://spotbugs.github.io/spotbugs-maven-plugin/plugin-info.html
SpotBugs Rules Reference(s):
- Find Security Bugs Reference: https://find-sec-bugs.github.io/bugs.htm
- SpotBugs Reference: https://spotbugs.readthedocs.io/en/latest/bugDescriptions.html
**For more information on PMD:**
- Maven PMD Plugin Docs: https://maven.apache.org/plugins/maven-pmd-plugin/check-mojo.html
- PMD Rule Reference: https://docs.pmd-code.org/latest/pmd_rules_java.html
## Troubleshooting
### Problem
If your db service continuously restarts and when looking at the logs the message indicates:
```
initdb: error: directory "/var/lib/postgresql/data" exists but is not empty.....
```
### Solution
1. Navigate to `/direct-file/direct-file/docker/db/postgres`
2. `rm -rf data`
3. Navigate back to `direct-file/direct-file/backend`
4. try `docker compose up -d`

7
direct-file/backend/docker/localstack/feature-flags.json Normal file
View file

@ -0,0 +1,7 @@
{
"open-enrollment": {
"new-users-allowed": true,
"max-users": 200000000
},
"esignature-enabled": true
}

39
direct-file/backend/docker_build.sh Executable file
View file

@ -0,0 +1,39 @@
#!/usr/bin/env bash
set -e
: "${MEF_REPO?Path to MeF SDK repo}"
SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd)
cd "$SCRIPT_DIR"
api_build_image_tag="direct-file-api-builder"
api_build_container_name="direct-file-api-builder-container"
api_jar_file_name="directfile-api-0.0.1-SNAPSHOT.jar"
jar_output_path_component="target"
# build jar
docker buildx build \
--pull \
--build-context factgraph-repo="../fact-graph-scala" \
--build-context boms="../boms" \
--build-context config="../config" \
--build-context shared-libs="../libs" \
--build-context scripts="../scripts" \
--build-context mef-sdk-repo="$MEF_REPO" \
--build-arg MAVEN_OPTS="$MAVEN_OPTS" \
--build-arg MAVEN_CLI_OPTS="$MAVEN_CLI_OPTS" \
--tag "$api_build_image_tag" \
--file Dockerfile-local \
--target api-builder \
"$@" \
"$SCRIPT_DIR"
# extract jar to `./target`
mkdir -p "$SCRIPT_DIR/$jar_output_path_component"
docker container rm --force "$api_build_container_name" &>/dev/null
docker container create --name "$api_build_container_name" "$api_build_image_tag"
docker cp "$api_build_container_name":/build/$jar_output_path_component/"$api_jar_file_name" "$SCRIPT_DIR/$jar_output_path_component/$api_jar_file_name"
docker container rm --force "$api_build_container_name"
printf "\njarfile: %s\n" "$SCRIPT_DIR/$jar_output_path_component/$api_jar_file_name"

259
direct-file/backend/mvnw vendored Executable file
View file

@ -0,0 +1,259 @@
#!/bin/sh
# ----------------------------------------------------------------------------
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
# ----------------------------------------------------------------------------
# ----------------------------------------------------------------------------
# Apache Maven Wrapper startup batch script, version 3.3.2
#
# Optional ENV vars
# -----------------
# JAVA_HOME - location of a JDK home dir, required when download maven via java source
# MVNW_REPOURL - repo url base for downloading maven distribution
# MVNW_USERNAME/MVNW_PASSWORD - user and password for downloading maven
# MVNW_VERBOSE - true: enable verbose log; debug: trace the mvnw script; others: silence the output
# ----------------------------------------------------------------------------
set -euf
[ "${MVNW_VERBOSE-}" != debug ] || set -x
# OS specific support.
native_path() { printf %s\\n "$1"; }
case "$(uname)" in
CYGWIN* | MINGW*)
[ -z "${JAVA_HOME-}" ] || JAVA_HOME="$(cygpath --unix "$JAVA_HOME")"
native_path() { cygpath --path --windows "$1"; }
;;
esac
# set JAVACMD and JAVACCMD
set_java_home() {
# For Cygwin and MinGW, ensure paths are in Unix format before anything is touched
if [ -n "${JAVA_HOME-}" ]; then
if [ -x "$JAVA_HOME/jre/sh/java" ]; then
# IBM's JDK on AIX uses strange locations for the executables
JAVACMD="$JAVA_HOME/jre/sh/java"
JAVACCMD="$JAVA_HOME/jre/sh/javac"
else
JAVACMD="$JAVA_HOME/bin/java"
JAVACCMD="$JAVA_HOME/bin/javac"
if [ ! -x "$JAVACMD" ] || [ ! -x "$JAVACCMD" ]; then
echo "The JAVA_HOME environment variable is not defined correctly, so mvnw cannot run." >&2
echo "JAVA_HOME is set to \"$JAVA_HOME\", but \"\$JAVA_HOME/bin/java\" or \"\$JAVA_HOME/bin/javac\" does not exist." >&2
return 1
fi
fi
else
JAVACMD="$(
'set' +e
'unset' -f command 2>/dev/null
'command' -v java
)" || :
JAVACCMD="$(
'set' +e
'unset' -f command 2>/dev/null
'command' -v javac
)" || :
if [ ! -x "${JAVACMD-}" ] || [ ! -x "${JAVACCMD-}" ]; then
echo "The java/javac command does not exist in PATH nor is JAVA_HOME set, so mvnw cannot run." >&2
return 1
fi
fi
}
# hash string like Java String::hashCode
hash_string() {
str="${1:-}" h=0
while [ -n "$str" ]; do
char="${str%"${str#?}"}"
h=$(((h * 31 + $(LC_CTYPE=C printf %d "'$char")) % 4294967296))
str="${str#?}"
done
printf %x\\n $h
}
verbose() { :; }
[ "${MVNW_VERBOSE-}" != true ] || verbose() { printf %s\\n "${1-}"; }
die() {
printf %s\\n "$1" >&2
exit 1
}
trim() {
# MWRAPPER-139:
# Trims trailing and leading whitespace, carriage returns, tabs, and linefeeds.
# Needed for removing poorly interpreted newline sequences when running in more
# exotic environments such as mingw bash on Windows.
printf "%s" "${1}" | tr -d '[:space:]'
}
# parse distributionUrl and optional distributionSha256Sum, requires .mvn/wrapper/maven-wrapper.properties
while IFS="=" read -r key value; do
case "${key-}" in
distributionUrl) distributionUrl=$(trim "${value-}") ;;
distributionSha256Sum) distributionSha256Sum=$(trim "${value-}") ;;
esac
done <"${0%/*}/.mvn/wrapper/maven-wrapper.properties"
[ -n "${distributionUrl-}" ] || die "cannot read distributionUrl property in ${0%/*}/.mvn/wrapper/maven-wrapper.properties"
case "${distributionUrl##*/}" in
maven-mvnd-*bin.*)
MVN_CMD=mvnd.sh _MVNW_REPO_PATTERN=/maven/mvnd/
case "${PROCESSOR_ARCHITECTURE-}${PROCESSOR_ARCHITEW6432-}:$(uname -a)" in
*AMD64:CYGWIN* | *AMD64:MINGW*) distributionPlatform=windows-amd64 ;;
:Darwin*x86_64) distributionPlatform=darwin-amd64 ;;
:Darwin*arm64) distributionPlatform=darwin-aarch64 ;;
:Linux*x86_64*) distributionPlatform=linux-amd64 ;;
*)
echo "Cannot detect native platform for mvnd on $(uname)-$(uname -m), use pure java version" >&2
distributionPlatform=linux-amd64
;;
esac
distributionUrl="${distributionUrl%-bin.*}-$distributionPlatform.zip"
;;
maven-mvnd-*) MVN_CMD=mvnd.sh _MVNW_REPO_PATTERN=/maven/mvnd/ ;;
*) MVN_CMD="mvn${0##*/mvnw}" _MVNW_REPO_PATTERN=/org/apache/maven/ ;;
esac
# apply MVNW_REPOURL and calculate MAVEN_HOME
# maven home pattern: ~/.m2/wrapper/dists/{apache-maven-<version>,maven-mvnd-<version>-<platform>}/<hash>
[ -z "${MVNW_REPOURL-}" ] || distributionUrl="$MVNW_REPOURL$_MVNW_REPO_PATTERN${distributionUrl#*"$_MVNW_REPO_PATTERN"}"
distributionUrlName="${distributionUrl##*/}"
distributionUrlNameMain="${distributionUrlName%.*}"
distributionUrlNameMain="${distributionUrlNameMain%-bin}"
MAVEN_USER_HOME="${MAVEN_USER_HOME:-${HOME}/.m2}"
MAVEN_HOME="${MAVEN_USER_HOME}/wrapper/dists/${distributionUrlNameMain-}/$(hash_string "$distributionUrl")"
exec_maven() {
unset MVNW_VERBOSE MVNW_USERNAME MVNW_PASSWORD MVNW_REPOURL || :
exec "$MAVEN_HOME/bin/$MVN_CMD" "$@" || die "cannot exec $MAVEN_HOME/bin/$MVN_CMD"
}
if [ -d "$MAVEN_HOME" ]; then
verbose "found existing MAVEN_HOME at $MAVEN_HOME"
exec_maven "$@"
fi
case "${distributionUrl-}" in
*?-bin.zip | *?maven-mvnd-?*-?*.zip) ;;
*) die "distributionUrl is not valid, must match *-bin.zip or maven-mvnd-*.zip, but found '${distributionUrl-}'" ;;
esac
# prepare tmp dir
if TMP_DOWNLOAD_DIR="$(mktemp -d)" && [ -d "$TMP_DOWNLOAD_DIR" ]; then
clean() { rm -rf -- "$TMP_DOWNLOAD_DIR"; }
trap clean HUP INT TERM EXIT
else
die "cannot create temp dir"
fi
mkdir -p -- "${MAVEN_HOME%/*}"
# Download and Install Apache Maven
verbose "Couldn't find MAVEN_HOME, downloading and installing it ..."
verbose "Downloading from: $distributionUrl"
verbose "Downloading to: $TMP_DOWNLOAD_DIR/$distributionUrlName"
# select .zip or .tar.gz
if ! command -v unzip >/dev/null; then
distributionUrl="${distributionUrl%.zip}.tar.gz"
distributionUrlName="${distributionUrl##*/}"
fi
# verbose opt
__MVNW_QUIET_WGET=--quiet __MVNW_QUIET_CURL=--silent __MVNW_QUIET_UNZIP=-q __MVNW_QUIET_TAR=''
[ "${MVNW_VERBOSE-}" != true ] || __MVNW_QUIET_WGET='' __MVNW_QUIET_CURL='' __MVNW_QUIET_UNZIP='' __MVNW_QUIET_TAR=v
# normalize http auth
case "${MVNW_PASSWORD:+has-password}" in
'') MVNW_USERNAME='' MVNW_PASSWORD='' ;;
has-password) [ -n "${MVNW_USERNAME-}" ] || MVNW_USERNAME='' MVNW_PASSWORD='' ;;
esac
if [ -z "${MVNW_USERNAME-}" ] && command -v wget >/dev/null; then
verbose "Found wget ... using wget"
wget ${__MVNW_QUIET_WGET:+"$__MVNW_QUIET_WGET"} "$distributionUrl" -O "$TMP_DOWNLOAD_DIR/$distributionUrlName" || die "wget: Failed to fetch $distributionUrl"
elif [ -z "${MVNW_USERNAME-}" ] && command -v curl >/dev/null; then
verbose "Found curl ... using curl"
curl ${__MVNW_QUIET_CURL:+"$__MVNW_QUIET_CURL"} -f -L -o "$TMP_DOWNLOAD_DIR/$distributionUrlName" "$distributionUrl" || die "curl: Failed to fetch $distributionUrl"
elif set_java_home; then
verbose "Falling back to use Java to download"
javaSource="$TMP_DOWNLOAD_DIR/Downloader.java"
targetZip="$TMP_DOWNLOAD_DIR/$distributionUrlName"
cat >"$javaSource" <<-END
public class Downloader extends java.net.Authenticator
{
protected java.net.PasswordAuthentication getPasswordAuthentication()
{
return new java.net.PasswordAuthentication( System.getenv( "MVNW_USERNAME" ), System.getenv( "MVNW_PASSWORD" ).toCharArray() );
}
public static void main( String[] args ) throws Exception
{
setDefault( new Downloader() );
java.nio.file.Files.copy( java.net.URI.create( args[0] ).toURL().openStream(), java.nio.file.Paths.get( args[1] ).toAbsolutePath().normalize() );
}
}
END
# For Cygwin/MinGW, switch paths to Windows format before running javac and java
verbose " - Compiling Downloader.java ..."
"$(native_path "$JAVACCMD")" "$(native_path "$javaSource")" || die "Failed to compile Downloader.java"
verbose " - Running Downloader.java ..."
"$(native_path "$JAVACMD")" -cp "$(native_path "$TMP_DOWNLOAD_DIR")" Downloader "$distributionUrl" "$(native_path "$targetZip")"
fi
# If specified, validate the SHA-256 sum of the Maven distribution zip file
if [ -n "${distributionSha256Sum-}" ]; then
distributionSha256Result=false
if [ "$MVN_CMD" = mvnd.sh ]; then
echo "Checksum validation is not supported for maven-mvnd." >&2
echo "Please disable validation by removing 'distributionSha256Sum' from your maven-wrapper.properties." >&2
exit 1
elif command -v sha256sum >/dev/null; then
if echo "$distributionSha256Sum $TMP_DOWNLOAD_DIR/$distributionUrlName" | sha256sum -c >/dev/null 2>&1; then
distributionSha256Result=true
fi
elif command -v shasum >/dev/null; then
if echo "$distributionSha256Sum $TMP_DOWNLOAD_DIR/$distributionUrlName" | shasum -a 256 -c >/dev/null 2>&1; then
distributionSha256Result=true
fi
else
echo "Checksum validation was requested but neither 'sha256sum' or 'shasum' are available." >&2
echo "Please install either command, or disable validation by removing 'distributionSha256Sum' from your maven-wrapper.properties." >&2
exit 1
fi
if [ $distributionSha256Result = false ]; then
echo "Error: Failed to validate Maven distribution SHA-256, your Maven distribution might be compromised." >&2
echo "If you updated your Maven version, you need to update the specified distributionSha256Sum property." >&2
exit 1
fi
fi
# unzip and move
if command -v unzip >/dev/null; then
unzip ${__MVNW_QUIET_UNZIP:+"$__MVNW_QUIET_UNZIP"} "$TMP_DOWNLOAD_DIR/$distributionUrlName" -d "$TMP_DOWNLOAD_DIR" || die "failed to unzip"
else
tar xzf${__MVNW_QUIET_TAR:+"$__MVNW_QUIET_TAR"} "$TMP_DOWNLOAD_DIR/$distributionUrlName" -C "$TMP_DOWNLOAD_DIR" || die "failed to untar"
fi
printf %s\\n "$distributionUrl" >"$TMP_DOWNLOAD_DIR/$distributionUrlNameMain/mvnw.url"
mv -- "$TMP_DOWNLOAD_DIR/$distributionUrlNameMain" "$MAVEN_HOME" || [ -d "$MAVEN_HOME" ] || die "fail to move MAVEN_HOME"
clean || :
exec_maven "$@"

149
direct-file/backend/mvnw.cmd vendored Normal file
View file

@ -0,0 +1,149 @@
<# : batch portion
@REM ----------------------------------------------------------------------------
@REM Licensed to the Apache Software Foundation (ASF) under one
@REM or more contributor license agreements. See the NOTICE file
@REM distributed with this work for additional information
@REM regarding copyright ownership. The ASF licenses this file
@REM to you under the Apache License, Version 2.0 (the
@REM "License"); you may not use this file except in compliance
@REM with the License. You may obtain a copy of the License at
@REM
@REM http://www.apache.org/licenses/LICENSE-2.0
@REM
@REM Unless required by applicable law or agreed to in writing,
@REM software distributed under the License is distributed on an
@REM "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@REM KIND, either express or implied. See the License for the
@REM specific language governing permissions and limitations
@REM under the License.
@REM ----------------------------------------------------------------------------
@REM ----------------------------------------------------------------------------
@REM Apache Maven Wrapper startup batch script, version 3.3.2
@REM
@REM Optional ENV vars
@REM MVNW_REPOURL - repo url base for downloading maven distribution
@REM MVNW_USERNAME/MVNW_PASSWORD - user and password for downloading maven
@REM MVNW_VERBOSE - true: enable verbose log; others: silence the output
@REM ----------------------------------------------------------------------------
@IF "%__MVNW_ARG0_NAME__%"=="" (SET __MVNW_ARG0_NAME__=%~nx0)
@SET __MVNW_CMD__=
@SET __MVNW_ERROR__=
@SET __MVNW_PSMODULEP_SAVE=%PSModulePath%
@SET PSModulePath=
@FOR /F "usebackq tokens=1* delims==" %%A IN (`powershell -noprofile "& {$scriptDir='%~dp0'; $script='%__MVNW_ARG0_NAME__%'; icm -ScriptBlock ([Scriptblock]::Create((Get-Content -Raw '%~f0'))) -NoNewScope}"`) DO @(
IF "%%A"=="MVN_CMD" (set __MVNW_CMD__=%%B) ELSE IF "%%B"=="" (echo %%A) ELSE (echo %%A=%%B)
)
@SET PSModulePath=%__MVNW_PSMODULEP_SAVE%
@SET __MVNW_PSMODULEP_SAVE=
@SET __MVNW_ARG0_NAME__=
@SET MVNW_USERNAME=
@SET MVNW_PASSWORD=
@IF NOT "%__MVNW_CMD__%"=="" (%__MVNW_CMD__% %*)
@echo Cannot start maven from wrapper >&2 && exit /b 1
@GOTO :EOF
: end batch / begin powershell #>
$ErrorActionPreference = "Stop"
if ($env:MVNW_VERBOSE -eq "true") {
$VerbosePreference = "Continue"
}
# calculate distributionUrl, requires .mvn/wrapper/maven-wrapper.properties
$distributionUrl = (Get-Content -Raw "$scriptDir/.mvn/wrapper/maven-wrapper.properties" | ConvertFrom-StringData).distributionUrl
if (!$distributionUrl) {
Write-Error "cannot read distributionUrl property in $scriptDir/.mvn/wrapper/maven-wrapper.properties"
}
switch -wildcard -casesensitive ( $($distributionUrl -replace '^.*/','') ) {
"maven-mvnd-*" {
$USE_MVND = $true
$distributionUrl = $distributionUrl -replace '-bin\.[^.]*$',"-windows-amd64.zip"
$MVN_CMD = "mvnd.cmd"
break
}
default {
$USE_MVND = $false
$MVN_CMD = $script -replace '^mvnw','mvn'
break
}
}
# apply MVNW_REPOURL and calculate MAVEN_HOME
# maven home pattern: ~/.m2/wrapper/dists/{apache-maven-<version>,maven-mvnd-<version>-<platform>}/<hash>
if ($env:MVNW_REPOURL) {
$MVNW_REPO_PATTERN = if ($USE_MVND) { "/org/apache/maven/" } else { "/maven/mvnd/" }
$distributionUrl = "$env:MVNW_REPOURL$MVNW_REPO_PATTERN$($distributionUrl -replace '^.*'+$MVNW_REPO_PATTERN,'')"
}
$distributionUrlName = $distributionUrl -replace '^.*/',''
$distributionUrlNameMain = $distributionUrlName -replace '\.[^.]*$','' -replace '-bin$',''
$MAVEN_HOME_PARENT = "$HOME/.m2/wrapper/dists/$distributionUrlNameMain"
if ($env:MAVEN_USER_HOME) {
$MAVEN_HOME_PARENT = "$env:MAVEN_USER_HOME/wrapper/dists/$distributionUrlNameMain"
}
$MAVEN_HOME_NAME = ([System.Security.Cryptography.MD5]::Create().ComputeHash([byte[]][char[]]$distributionUrl) | ForEach-Object {$_.ToString("x2")}) -join ''
$MAVEN_HOME = "$MAVEN_HOME_PARENT/$MAVEN_HOME_NAME"
if (Test-Path -Path "$MAVEN_HOME" -PathType Container) {
Write-Verbose "found existing MAVEN_HOME at $MAVEN_HOME"
Write-Output "MVN_CMD=$MAVEN_HOME/bin/$MVN_CMD"
exit $?
}
if (! $distributionUrlNameMain -or ($distributionUrlName -eq $distributionUrlNameMain)) {
Write-Error "distributionUrl is not valid, must end with *-bin.zip, but found $distributionUrl"
}
# prepare tmp dir
$TMP_DOWNLOAD_DIR_HOLDER = New-TemporaryFile
$TMP_DOWNLOAD_DIR = New-Item -Itemtype Directory -Path "$TMP_DOWNLOAD_DIR_HOLDER.dir"
$TMP_DOWNLOAD_DIR_HOLDER.Delete() | Out-Null
trap {
if ($TMP_DOWNLOAD_DIR.Exists) {
try { Remove-Item $TMP_DOWNLOAD_DIR -Recurse -Force | Out-Null }
catch { Write-Warning "Cannot remove $TMP_DOWNLOAD_DIR" }
}
}
New-Item -Itemtype Directory -Path "$MAVEN_HOME_PARENT" -Force | Out-Null
# Download and Install Apache Maven
Write-Verbose "Couldn't find MAVEN_HOME, downloading and installing it ..."
Write-Verbose "Downloading from: $distributionUrl"
Write-Verbose "Downloading to: $TMP_DOWNLOAD_DIR/$distributionUrlName"
$webclient = New-Object System.Net.WebClient
if ($env:MVNW_USERNAME -and $env:MVNW_PASSWORD) {
$webclient.Credentials = New-Object System.Net.NetworkCredential($env:MVNW_USERNAME, $env:MVNW_PASSWORD)
}
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$webclient.DownloadFile($distributionUrl, "$TMP_DOWNLOAD_DIR/$distributionUrlName") | Out-Null
# If specified, validate the SHA-256 sum of the Maven distribution zip file
$distributionSha256Sum = (Get-Content -Raw "$scriptDir/.mvn/wrapper/maven-wrapper.properties" | ConvertFrom-StringData).distributionSha256Sum
if ($distributionSha256Sum) {
if ($USE_MVND) {
Write-Error "Checksum validation is not supported for maven-mvnd. `nPlease disable validation by removing 'distributionSha256Sum' from your maven-wrapper.properties."
}
Import-Module $PSHOME\Modules\Microsoft.PowerShell.Utility -Function Get-FileHash
if ((Get-FileHash "$TMP_DOWNLOAD_DIR/$distributionUrlName" -Algorithm SHA256).Hash.ToLower() -ne $distributionSha256Sum) {
Write-Error "Error: Failed to validate Maven distribution SHA-256, your Maven distribution might be compromised. If you updated your Maven version, you need to update the specified distributionSha256Sum property."
}
}
# unzip and move
Expand-Archive "$TMP_DOWNLOAD_DIR/$distributionUrlName" -DestinationPath "$TMP_DOWNLOAD_DIR" | Out-Null
Rename-Item -Path "$TMP_DOWNLOAD_DIR/$distributionUrlNameMain" -NewName $MAVEN_HOME_NAME | Out-Null
try {
Move-Item -Path "$TMP_DOWNLOAD_DIR/$MAVEN_HOME_NAME" -Destination $MAVEN_HOME_PARENT | Out-Null
} catch {
if (! (Test-Path -Path "$MAVEN_HOME" -PathType Container)) {
Write-Error "fail to move MAVEN_HOME"
}
} finally {
try { Remove-Item $TMP_DOWNLOAD_DIR -Recurse -Force | Out-Null }
catch { Write-Warning "Cannot remove $TMP_DOWNLOAD_DIR" }
}
Write-Output "MVN_CMD=$MAVEN_HOME/bin/$MVN_CMD"

226
direct-file/backend/performance-tests/create-and-update-tax-return.jmx Normal file
View file

@ -0,0 +1,226 @@
<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="5.0" jmeter="5.6.3">
<hashTree>
<TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="Test Plan">
<elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables">
<collectionProp name="Arguments.arguments"/>
</elementProp>
<boolProp name="TestPlan.functional_mode">false</boolProp>
<boolProp name="TestPlan.serialize_threadgroups">false</boolProp>
</TestPlan>
<hashTree>
<SetupThreadGroup guiclass="SetupThreadGroupGui" testclass="SetupThreadGroup" testname="setUp Thread Group">
<intProp name="ThreadGroup.num_threads">1</intProp>
<intProp name="ThreadGroup.ramp_time">1</intProp>
<boolProp name="ThreadGroup.same_user_on_next_iteration">true</boolProp>
<stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
<elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller">
<stringProp name="LoopController.loops">1</stringProp>
<boolProp name="LoopController.continue_forever">false</boolProp>
</elementProp>
</SetupThreadGroup>
<hashTree>
<Arguments guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables">
<collectionProp name="Arguments.arguments">
<elementProp name="TAX_YEAR" elementType="Argument">
<stringProp name="Argument.name">TAX_YEAR</stringProp>
<stringProp name="Argument.value">2023</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
</collectionProp>
</Arguments>
<hashTree/>
<JSR223Sampler guiclass="TestBeanGUI" testclass="JSR223Sampler" testname="Set Resources Directory">
<stringProp name="cacheKey">true</stringProp>
<stringProp name="filename"></stringProp>
<stringProp name="parameters"></stringProp>
<stringProp name="script">import org.apache.jmeter.services.FileServer
// Get the base directory
String baseDirectory = FileServer.getFileServer().getBaseDir()
// Create a File object for the base directory
File baseDirectoryFile = new File(baseDirectory)
// Get the parent directory
// this should point to {your-git-repo-location}/direct-file/direct-file/backend
String backendDirectory = baseDirectoryFile.getParent()
String resourcesDirectory = backendDirectory + &quot;/performance-tests/resources&quot;;
// props are global, vars are thread-level
props.put(&quot;BACKEND_DIRECTORY&quot;, backendDirectory)
props.put(&quot;RESOURCES_DIRECTORY&quot;, resourcesDirectory)
log.info(&quot;Set RESOURCES_DIRECTORY prop to: &quot; + props.get(&quot;RESOURCES_DIRECTORY&quot;))</stringProp>
<stringProp name="scriptLanguage">groovy</stringProp>
</JSR223Sampler>
<hashTree/>
</hashTree>
<ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="Create and Update Tax Return">
<intProp name="ThreadGroup.num_threads">3</intProp>
<intProp name="ThreadGroup.ramp_time">1</intProp>
<boolProp name="ThreadGroup.same_user_on_next_iteration">true</boolProp>
<stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
<elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller">
<stringProp name="LoopController.loops">1</stringProp>
<boolProp name="LoopController.continue_forever">false</boolProp>
</elementProp>
</ThreadGroup>
<hashTree>
<UserParameters guiclass="UserParametersGui" testclass="UserParameters" testname="User Parameters">
<collectionProp name="UserParameters.names">
<stringProp name="1503496487">sadi_uuid</stringProp>
</collectionProp>
<collectionProp name="UserParameters.thread_values">
<collectionProp name="681405977">
<stringProp name="118040362">${__UUID()}</stringProp>
</collectionProp>
</collectionProp>
<boolProp name="UserParameters.per_iteration">true</boolProp>
</UserParameters>
<hashTree/>
<Arguments guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables">
<collectionProp name="Arguments.arguments">
<elementProp name="BACKEND_URI" elementType="Argument">
<stringProp name="Argument.name">BACKEND_URI</stringProp>
<stringProp name="Argument.value">localhost:8080</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
<elementProp name="X_FORWARDED_FOR_HEADER_VALUE" elementType="Argument">
<stringProp name="Argument.name">X_FORWARDED_FOR_HEADER_VALUE</stringProp>
<stringProp name="Argument.value">76.122.220.120</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
<elementProp name="CREATE_TAX_RETURN_JSON_REQUEST_BODY" elementType="Argument">
<stringProp name="Argument.name">CREATE_TAX_RETURN_JSON_REQUEST_BODY</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
</collectionProp>
</Arguments>
<hashTree/>
<JSR223PreProcessor guiclass="TestBeanGUI" testclass="JSR223PreProcessor" testname="Set UPDATE_TAX_RETURN_JSON_REQUEST_BODY">
<stringProp name="scriptLanguage">groovy</stringProp>
<stringProp name="parameters"></stringProp>
<stringProp name="filename"></stringProp>
<stringProp name="cacheKey">true</stringProp>
<stringProp name="script">import groovy.json.JsonSlurper
import groovy.json.JsonOutput
// Specify the file path
String resourcesDirectory = props.get(&quot;RESOURCES_DIRECTORY&quot;);
String filePath = resourcesDirectory + &quot;/update-tax-return-payloads/ats-1.json&quot;
// Read the file content
File file = new File(filePath)
String fileContent = file.text
vars.put(&quot;UPDATE_TAX_RETURN_JSON_REQUEST_BODY&quot;, fileContent);</stringProp>
</JSR223PreProcessor>
<hashTree/>
<HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="HTTP Header Manager">
<collectionProp name="HeaderManager.headers">
<elementProp name="" elementType="Header">
<stringProp name="Header.name">SM_UNIVERSALID</stringProp>
<stringProp name="Header.value">${sadi_uuid}</stringProp>
</elementProp>
<elementProp name="" elementType="Header">
<stringProp name="Header.name">Content-Type</stringProp>
<stringProp name="Header.value">application/json</stringProp>
</elementProp>
<elementProp name="" elementType="Header">
<stringProp name="Header.name">X-Forwarded-For</stringProp>
<stringProp name="Header.value">${X_FORWARDED_FOR_HEADER_VALUE}</stringProp>
</elementProp>
</collectionProp>
</HeaderManager>
<hashTree/>
<HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="Create New Tax Return">
<stringProp name="HTTPSampler.domain">localhost</stringProp>
<stringProp name="HTTPSampler.port">8080</stringProp>
<stringProp name="HTTPSampler.protocol">http</stringProp>
<stringProp name="HTTPSampler.path">/df/file/api/v1/taxreturns</stringProp>
<boolProp name="HTTPSampler.follow_redirects">true</boolProp>
<stringProp name="HTTPSampler.method">POST</stringProp>
<boolProp name="HTTPSampler.use_keepalive">true</boolProp>
<boolProp name="HTTPSampler.postBodyRaw">true</boolProp>
<elementProp name="HTTPsampler.Arguments" elementType="Arguments">
<collectionProp name="Arguments.arguments">
<elementProp name="" elementType="HTTPArgument">
<boolProp name="HTTPArgument.always_encode">false</boolProp>
<stringProp name="Argument.value">${CREATE_TAX_RETURN_JSON_REQUEST_BODY}</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
</collectionProp>
</elementProp>
</HTTPSamplerProxy>
<hashTree>
<JSONPostProcessor guiclass="JSONPostProcessorGui" testclass="JSONPostProcessor" testname="JSON Extractor" enabled="true">
<stringProp name="JSONPostProcessor.referenceNames">taxReturnId</stringProp>
<stringProp name="JSONPostProcessor.jsonPathExprs">$.id</stringProp>
<stringProp name="JSONPostProcessor.match_numbers"></stringProp>
<stringProp name="TestPlan.comments">Extract the taxReturnId from the create response body so that it can be used in a later update request</stringProp>
</JSONPostProcessor>
<hashTree/>
</hashTree>
<HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="Update Tax Return">
<stringProp name="HTTPSampler.domain">localhost</stringProp>
<stringProp name="HTTPSampler.port">8080</stringProp>
<stringProp name="HTTPSampler.protocol">http</stringProp>
<stringProp name="HTTPSampler.path">/df/file/api/v1/taxreturns/${taxReturnId}</stringProp>
<boolProp name="HTTPSampler.follow_redirects">true</boolProp>
<stringProp name="HTTPSampler.method">POST</stringProp>
<boolProp name="HTTPSampler.use_keepalive">true</boolProp>
<boolProp name="HTTPSampler.postBodyRaw">true</boolProp>
<elementProp name="HTTPsampler.Arguments" elementType="Arguments">
<collectionProp name="Arguments.arguments">
<elementProp name="" elementType="HTTPArgument">
<boolProp name="HTTPArgument.always_encode">false</boolProp>
<stringProp name="Argument.value">${UPDATE_TAX_RETURN_JSON_REQUEST_BODY}</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
</collectionProp>
</elementProp>
</HTTPSamplerProxy>
<hashTree/>
<ResultCollector guiclass="ViewResultsFullVisualizer" testclass="ResultCollector" testname="View Results Tree">
<boolProp name="ResultCollector.error_logging">false</boolProp>
<objProp>
<name>saveConfig</name>
<value class="SampleSaveConfiguration">
<time>true</time>
<latency>true</latency>
<timestamp>true</timestamp>
<success>true</success>
<label>true</label>
<code>true</code>
<message>true</message>
<threadName>true</threadName>
<dataType>true</dataType>
<encoding>false</encoding>
<assertions>true</assertions>
<subresults>true</subresults>
<responseData>false</responseData>
<samplerData>false</samplerData>
<xml>false</xml>
<fieldNames>true</fieldNames>
<responseHeaders>false</responseHeaders>
<requestHeaders>false</requestHeaders>
<responseDataOnError>false</responseDataOnError>
<saveAssertionResultsFailureMessage>true</saveAssertionResultsFailureMessage>
<assertionsResultsToSave>0</assertionsResultsToSave>
<bytes>true</bytes>
<sentBytes>true</sentBytes>
<url>true</url>
<threadCounts>true</threadCounts>
<idleTime>true</idleTime>
<connectTime>true</connectTime>
</value>
</objProp>
<stringProp name="filename"></stringProp>
</ResultCollector>
<hashTree/>
</hashTree>
</hashTree>
</hashTree>
</jmeterTestPlan>

30
direct-file/backend/performance-tests/readme.md Normal file
View file

@ -0,0 +1,30 @@
**Install Jmeter**
- ```brew install jmeter```
- ```jmeter -v```
**Start JMeter GUI**
- open terminal
- run command ```jmeter```
**Running Jmeter Test via GUI**
- start jmeter GUI
- open the desired ```.jmx``` file
- press the green play button
**Open Log Viewer in GUI**
- From the menu bar, enable the LogViewer via: ```Options -> LogViewer```
**Troubleshooting**
- If logs stop appearing in the console, restart JMeter.
**Props vs Vars**
- ```props``` are global key-value pairs that can be shared between threads.
- ```vars``` are thread-level key-value pairs that are cannot be shared between other threads.
**User Defined Variables vs User Parameters**
- User Defined Variables are set globally, and can be accessed by all threads.
- User Parameters are recomputed for every thread or optionally for very iteration
**Open Issues**
- for some reason, when I close JMeter and reopen it, and then run a test, the variable fails to evaluate

451
direct-file/backend/performance-tests/resources/update-tax-return-payloads/ats-1.json Normal file
View file

@ -0,0 +1,451 @@
{
"facts": {
"/address": {
"$type": "gov.irs.factgraph.persisters.AddressWrapper",
"item": {
"city": "Monroe",
"country": "",
"postalCode": "02301",
"stateOrProvence": "MA",
"streetAddress": "2030 Pecan Street"
}
},
"/familyAndHousehold": {
"$type": "gov.irs.factgraph.persisters.CollectionWrapper",
"item": { "items": [] }
},
"/dependentsIsDone": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": true
},
"/disposedDigitalAssets": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": false
},
"/email": {
"$type": "gov.irs.factgraph.persisters.EmailAddressWrapper",
"item": { "email": "hello.p.gov@test.test.com" }
},
"/filedLastYear": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": true
},
"/filerResidenceAndIncomeState": {
"$type": "gov.irs.factgraph.persisters.EnumWrapper",
"item": { "enumOptionsPath": "/scopedStateOptions", "value": ["ma"] }
},
"/filers": {
"$type": "gov.irs.factgraph.persisters.CollectionWrapper",
"item": {
"items": ["363812c5-2b5d-46fd-a1f7-49f07bcf59d7", "3d12941a-30be-4ca5-b5df-f7252f6ed8d2"]
}
},
"/filers/#363812c5-2b5d-46fd-a1f7-49f07bcf59d7/canBeClaimed": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": false
},
"/filers/#363812c5-2b5d-46fd-a1f7-49f07bcf59d7/dateOfBirth": {
"$type": "gov.irs.factgraph.persisters.DayWrapper",
"item": { "date": "1984-01-26" }
},
"/filers/#363812c5-2b5d-46fd-a1f7-49f07bcf59d7/firstName": {
"$type": "gov.irs.factgraph.persisters.StringWrapper",
"item": "Susan"
},
"/filers/#363812c5-2b5d-46fd-a1f7-49f07bcf59d7/hasIpPin": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": false
},
"/filers/#363812c5-2b5d-46fd-a1f7-49f07bcf59d7/isBlind": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": false
},
"/filers/#363812c5-2b5d-46fd-a1f7-49f07bcf59d7/isPrimaryFiler": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": true
},
"/filers/#363812c5-2b5d-46fd-a1f7-49f07bcf59d7/isUsCitizenFullYear": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": true
},
"/filers/#363812c5-2b5d-46fd-a1f7-49f07bcf59d7/lastName": {
"$type": "gov.irs.factgraph.persisters.StringWrapper",
"item": "Miranda"
},
"/filers/#363812c5-2b5d-46fd-a1f7-49f07bcf59d7/occupation": {
"$type": "gov.irs.factgraph.persisters.StringWrapper",
"item": "Scenario Tester"
},
"/filers/#363812c5-2b5d-46fd-a1f7-49f07bcf59d7/tin": {
"$type": "gov.irs.factgraph.persisters.TinWrapper",
"item": { "area": "400", "group": "00", "serial": "1032" }
},
"/filers/#3d12941a-30be-4ca5-b5df-f7252f6ed8d2/isPrimaryFiler": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": false
},
"/filingStatus": {
"$type": "gov.irs.factgraph.persisters.EnumWrapper",
"item": { "enumOptionsPath": "/filingStatusOptions", "value": ["single"] }
},
"/flowHasSeenAmount": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": true
},
"/flowHasSeenCreditsIntroNoCredits": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": true
},
"/flowHasSeenDeductions": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": true
},
"/form1099Gs": {
"$type": "gov.irs.factgraph.persisters.CollectionWrapper",
"item": { "items": [] }
},
"/form1099GsIsDone": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": true
},
"/formW2s": {
"$type": "gov.irs.factgraph.persisters.CollectionWrapper",
"item": {
"items": ["a6ab6f70-0282-4f05-8888-0288526e1ed2", "abeb6660-6832-4acb-aa99-34febd590ca9"]
}
},
"/formW2s/#a6ab6f70-0282-4f05-8888-0288526e1ed2/addressMatchesReturn": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": false
},
"/formW2s/#a6ab6f70-0282-4f05-8888-0288526e1ed2/addressOverride": {
"$type": "gov.irs.factgraph.persisters.AddressWrapper",
"item": {
"city": "Monroe",
"country": "",
"postalCode": "70201",
"stateOrProvence": "MA",
"streetAddress": "2030 Pecan Street"
}
},
"/formW2s/#a6ab6f70-0282-4f05-8888-0288526e1ed2/ein": {
"$type": "gov.irs.factgraph.persisters.EinWrapper",
"item": { "prefix": "00", "serial": "0000004" }
},
"/formW2s/#a6ab6f70-0282-4f05-8888-0288526e1ed2/employerAddress": {
"$type": "gov.irs.factgraph.persisters.AddressWrapper",
"item": {
"city": "Monroe",
"country": "",
"postalCode": "70201",
"stateOrProvence": "MA",
"streetAddress": "2045 Pecan Street"
}
},
"/formW2s/#a6ab6f70-0282-4f05-8888-0288526e1ed2/employerName": {
"$type": "gov.irs.factgraph.persisters.StringWrapper",
"item": "Our Flower Shop"
},
"/formW2s/#a6ab6f70-0282-4f05-8888-0288526e1ed2/filer": {
"$type": "gov.irs.factgraph.persisters.CollectionItemWrapper",
"item": { "id": "363812c5-2b5d-46fd-a1f7-49f07bcf59d7" }
},
"/formW2s/#a6ab6f70-0282-4f05-8888-0288526e1ed2/hasRRTACodes": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": false
},
"/formW2s/#a6ab6f70-0282-4f05-8888-0288526e1ed2/hasSeenLastAvailableScreen": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": true
},
"/formW2s/#a6ab6f70-0282-4f05-8888-0288526e1ed2/nonstandardOrCorrectedChoice": {
"$type": "gov.irs.factgraph.persisters.EnumWrapper",
"item": {
"enumOptionsPath": "/w2NonstandardCorrectedOptions",
"value": ["neither"]
}
},
"/formW2s/#a6ab6f70-0282-4f05-8888-0288526e1ed2/retirementPlan": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": false
},
"/formW2s/#a6ab6f70-0282-4f05-8888-0288526e1ed2/statutoryEmployee": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": false
},
"/formW2s/#a6ab6f70-0282-4f05-8888-0288526e1ed2/thirdPartySickPay": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": false
},
"/formW2s/#a6ab6f70-0282-4f05-8888-0288526e1ed2/tin": {
"$type": "gov.irs.factgraph.persisters.TinWrapper",
"item": { "area": "400", "group": "00", "serial": "1032" }
},
"/formW2s/#a6ab6f70-0282-4f05-8888-0288526e1ed2/writableFederalWithholding": {
"$type": "gov.irs.factgraph.persisters.DollarWrapper",
"item": "1405.00"
},
"/formW2s/#a6ab6f70-0282-4f05-8888-0288526e1ed2/writableMedicareWages": {
"$type": "gov.irs.factgraph.persisters.DollarWrapper",
"item": "15205.00"
},
"/formW2s/#a6ab6f70-0282-4f05-8888-0288526e1ed2/writableMedicareWithholding": {
"$type": "gov.irs.factgraph.persisters.DollarWrapper",
"item": "220.00"
},
"/formW2s/#a6ab6f70-0282-4f05-8888-0288526e1ed2/writableOasdiWages": {
"$type": "gov.irs.factgraph.persisters.DollarWrapper",
"item": "15205.00"
},
"/formW2s/#a6ab6f70-0282-4f05-8888-0288526e1ed2/writableOasdiWithholding": {
"$type": "gov.irs.factgraph.persisters.DollarWrapper",
"item": "943.00"
},
"/formW2s/#a6ab6f70-0282-4f05-8888-0288526e1ed2/writableState": {
"$type": "gov.irs.factgraph.persisters.EnumWrapper",
"item": {
"enumOptionsPath": "/incomeFormStateOptions",
"value": ["sameState"]
}
},
"/formW2s/#a6ab6f70-0282-4f05-8888-0288526e1ed2/writableStateEmployerId": {
"$type": "gov.irs.factgraph.persisters.StringWrapper",
"item": "00-0000005"
},
"/formW2s/#a6ab6f70-0282-4f05-8888-0288526e1ed2/writableStateWages": {
"$type": "gov.irs.factgraph.persisters.DollarWrapper",
"item": "15205.00"
},
"/formW2s/#a6ab6f70-0282-4f05-8888-0288526e1ed2/writableStateWithholding": {
"$type": "gov.irs.factgraph.persisters.DollarWrapper",
"item": "507.00"
},
"/formW2s/#a6ab6f70-0282-4f05-8888-0288526e1ed2/writableWages": {
"$type": "gov.irs.factgraph.persisters.DollarWrapper",
"item": "15205.00"
},
"/formW2s/#abeb6660-6832-4acb-aa99-34febd590ca9/addressMatchesReturn": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": false
},
"/formW2s/#abeb6660-6832-4acb-aa99-34febd590ca9/addressOverride": {
"$type": "gov.irs.factgraph.persisters.AddressWrapper",
"item": {
"city": "Monroe",
"country": "",
"postalCode": "70201",
"stateOrProvence": "MA",
"streetAddress": "2030 Pecan Street"
}
},
"/formW2s/#abeb6660-6832-4acb-aa99-34febd590ca9/ein": {
"$type": "gov.irs.factgraph.persisters.EinWrapper",
"item": { "prefix": "00", "serial": "0000007" }
},
"/formW2s/#abeb6660-6832-4acb-aa99-34febd590ca9/employerAddress": {
"$type": "gov.irs.factgraph.persisters.AddressWrapper",
"item": {
"city": "Monroe",
"country": "",
"postalCode": "70201",
"stateOrProvence": "MA",
"streetAddress": "1001 Main Street"
}
},
"/formW2s/#abeb6660-6832-4acb-aa99-34febd590ca9/employerName": {
"$type": "gov.irs.factgraph.persisters.StringWrapper",
"item": "Magnolia Floral Design"
},
"/formW2s/#abeb6660-6832-4acb-aa99-34febd590ca9/filer": {
"$type": "gov.irs.factgraph.persisters.CollectionItemWrapper",
"item": { "id": "363812c5-2b5d-46fd-a1f7-49f07bcf59d7" }
},
"/formW2s/#abeb6660-6832-4acb-aa99-34febd590ca9/hasRRTACodes": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": false
},
"/formW2s/#abeb6660-6832-4acb-aa99-34febd590ca9/hasSeenLastAvailableScreen": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": true
},
"/formW2s/#abeb6660-6832-4acb-aa99-34febd590ca9/nonstandardOrCorrectedChoice": {
"$type": "gov.irs.factgraph.persisters.EnumWrapper",
"item": {
"enumOptionsPath": "/w2NonstandardCorrectedOptions",
"value": ["neither"]
}
},
"/formW2s/#abeb6660-6832-4acb-aa99-34febd590ca9/retirementPlan": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": false
},
"/formW2s/#abeb6660-6832-4acb-aa99-34febd590ca9/statutoryEmployee": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": false
},
"/formW2s/#abeb6660-6832-4acb-aa99-34febd590ca9/thirdPartySickPay": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": false
},
"/formW2s/#abeb6660-6832-4acb-aa99-34febd590ca9/tin": {
"$type": "gov.irs.factgraph.persisters.TinWrapper",
"item": { "area": "400", "group": "00", "serial": "1032" }
},
"/formW2s/#abeb6660-6832-4acb-aa99-34febd590ca9/writableFederalWithholding": {
"$type": "gov.irs.factgraph.persisters.DollarWrapper",
"item": "5869.00"
},
"/formW2s/#abeb6660-6832-4acb-aa99-34febd590ca9/writableMedicareWages": {
"$type": "gov.irs.factgraph.persisters.DollarWrapper",
"item": "24469.00"
},
"/formW2s/#abeb6660-6832-4acb-aa99-34febd590ca9/writableMedicareWithholding": {
"$type": "gov.irs.factgraph.persisters.DollarWrapper",
"item": "355.00"
},
"/formW2s/#abeb6660-6832-4acb-aa99-34febd590ca9/writableOasdiWages": {
"$type": "gov.irs.factgraph.persisters.DollarWrapper",
"item": "24469.00"
},
"/formW2s/#abeb6660-6832-4acb-aa99-34febd590ca9/writableOasdiWithholding": {
"$type": "gov.irs.factgraph.persisters.DollarWrapper",
"item": "1517.00"
},
"/formW2s/#abeb6660-6832-4acb-aa99-34febd590ca9/writableState": {
"$type": "gov.irs.factgraph.persisters.EnumWrapper",
"item": {
"enumOptionsPath": "/incomeFormStateOptions",
"value": ["sameState"]
}
},
"/formW2s/#abeb6660-6832-4acb-aa99-34febd590ca9/writableStateEmployerId": {
"$type": "gov.irs.factgraph.persisters.StringWrapper",
"item": "00-0000008"
},
"/formW2s/#abeb6660-6832-4acb-aa99-34febd590ca9/writableStateWages": {
"$type": "gov.irs.factgraph.persisters.DollarWrapper",
"item": "24469.00"
},
"/formW2s/#abeb6660-6832-4acb-aa99-34febd590ca9/writableStateWithholding": {
"$type": "gov.irs.factgraph.persisters.DollarWrapper",
"item": "1502.00"
},
"/formW2s/#abeb6660-6832-4acb-aa99-34febd590ca9/writableWages": {
"$type": "gov.irs.factgraph.persisters.DollarWrapper",
"item": "24469.00"
},
"/formW2sIsDone": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": true
},
"/hadStudentLoanInterestPayments": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": false
},
"/hasSeenReviewScreen": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": true
},
"/incomeFormsInScopedState": {
"$type": "gov.irs.factgraph.persisters.EnumWrapper",
"item": {
"enumOptionsPath": "/incomeStateOptions",
"value": ["onlySame"]
}
},
"/incomeSourcesSupported": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": true
},
"/interestReports": {
"$type": "gov.irs.factgraph.persisters.CollectionWrapper",
"item": { "items": [] }
},
"/interestReportsIsDone": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": true
},
"/lastYearAgi": {
"$type": "gov.irs.factgraph.persisters.DollarWrapper",
"item": "50.00"
},
"/maritalStatus": {
"$type": "gov.irs.factgraph.persisters.EnumWrapper",
"item": {
"enumOptionsPath": "/maritalStatusOptions",
"value": ["single"]
}
},
"/paidEstimatedTaxesOrFromLastYear": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": false
},
"/phone": {
"$type": "gov.irs.factgraph.persisters.E164Wrapper",
"item": {
"$type": "gov.irs.factgraph.types.UsPhoneNumber",
"areaCode": "202",
"lineNumber": "1212",
"officeCode": "555"
}
},
"/presidentalCampaignDesignation": {
"$type": "gov.irs.factgraph.persisters.EnumWrapper",
"item": {
"enumOptionsPath": "/presidentalCampaignOptions",
"value": ["neither"]
}
},
"/receivedDigitalAssets": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": false
},
"/refundViaAch": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": false
},
"/selfSelectPin": {
"$type": "gov.irs.factgraph.persisters.PinWrapper",
"item": { "pin": "20833" }
},
"/signReturnIdentity": {
"$type": "gov.irs.factgraph.persisters.EnumWrapper",
"item": {
"enumOptionsPath": "/signReturnIdentityOptions",
"value": ["lastYearAgi"]
}
},
"/socialSecurityReports": {
"$type": "gov.irs.factgraph.persisters.CollectionWrapper",
"item": { "items": [] }
},
"/socialSecurityReportsIsDone": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": true
},
"/wantsCommsFormat": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": false
},
"/wantsCustomLanguage": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": false
},
"/wantsThirdPartyDesignee": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": false
},
"/hasForeignAccounts": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": false
},
"/hasForeignTrusts": {
"$type": "gov.irs.factgraph.persisters.BooleanWrapper",
"item": false
},
"/wasK12Educators": {
"$type": "gov.irs.factgraph.persisters.EnumWrapper",
"item": { "enumOptionsPath": "/k12EducatorOptions", "value": ["neither"] }
}
}
}

284
direct-file/backend/pom.xml Normal file
View file

@ -0,0 +1,284 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>gov.irs.directfile.boot</groupId>
<artifactId>irs-spring-boot-starter-parent</artifactId>
<version>0.0.1-SNAPSHOT</version>
<relativePath>../boms/irs-spring-boot-starter-parent</relativePath>
</parent>
<groupId>gov.irs.directfile</groupId>
<artifactId>directfile-api</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>Direct File API</name>
<description>backend application for direct file</description>
<properties>
<!-- overrides for properties defined in irs-spring-boot-starter-parent -->
<config-folder.path>${project.basedir}/../config</config-folder.path>
</properties>
<dependencies>
<dependency>
<groupId>gov.irs.directfile</groupId>
<artifactId>data-models</artifactId>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
</dependency>
<dependency>
<groupId>javax.xml.bind</groupId>
<artifactId>jaxb-api</artifactId>
</dependency>
<dependency>
<groupId>org.apache.pdfbox</groupId>
<artifactId>pdfbox</artifactId>
<exclusions>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-validation</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-configuration-processor</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.wiremock</groupId>
<artifactId>wiremock-jetty12</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>gov.irs.factgraph</groupId>
<artifactId>fact-graph_3</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.liquibase</groupId>
<artifactId>liquibase-core</artifactId>
</dependency>
<dependency>
<groupId>org.modelmapper</groupId>
<artifactId>modelmapper</artifactId>
</dependency>
<!-- https://mvnrepository.com/artifact/org.modelmapper/modelmapper-module-record -->
<dependency>
<groupId>org.modelmapper</groupId>
<artifactId>modelmapper-module-record</artifactId>
</dependency>
<dependency>
<groupId>org.hibernate.orm</groupId>
<artifactId>hibernate-core</artifactId>
</dependency>
<dependency>
<groupId>org.springdoc</groupId>
<artifactId>springdoc-openapi-starter-webmvc-ui</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
</dependency>
<!-- https://mvnrepository.com/artifact/com.fasterxml.jackson.datatype/jackson-datatype-jsr310 -->
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jsr310</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcpkix-jdk18on</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>kms</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>sns</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>sqs</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>s3</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>sts</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.encryption.s3</groupId>
<artifactId>amazon-s3-encryption-client-java</artifactId>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>amazon-sqs-java-messaging-lib</artifactId>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>net.logstash.logback</groupId>
<artifactId>logstash-logback-encoder</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-compress</artifactId>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webflux</artifactId>
</dependency>
<dependency>
<groupId>com.github.spotbugs</groupId>
<artifactId>spotbugs-annotations</artifactId>
</dependency>
<dependency>
<groupId>jakarta.xml.ws</groupId>
<artifactId>jakarta.xml.ws-api</artifactId>
</dependency>
<dependency>
<groupId>com.sun.xml.bind</groupId>
<artifactId>jaxb-impl</artifactId>
</dependency>
<dependency>
<groupId>com.sun.xml.ws</groupId>
<artifactId>jaxws-rt</artifactId>
</dependency>
<dependency>
<groupId>com.nimbusds</groupId>
<artifactId>nimbus-jose-jwt</artifactId>
<version>9.37.3</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.retry</groupId>
<artifactId>spring-retry</artifactId>
</dependency>
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>okhttp</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>mockwebserver</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>-XX:+EnableDynamicAgentLoading ${argLine}</argLine>
<excludedEnvironmentVariables>
<!-- exclude active profiles so tests work from cli even if this var is set -->
<excludedEnvironmentVariable>SPRING_PROFILES_ACTIVE</excludedEnvironmentVariable>
</excludedEnvironmentVariables>
<excludes>
<!-- exclude end-to-end tests from `test` lifecycle phase -->
<exclude>**/*EndToEndTest</exclude>
<!-- exclude scenario tests from `test` lifecycle phase. These tests take a significant
amount of time to run, so we parallelize the running of these tests in CI. -->
<exclude>gov.irs.directfile.api.loaders.service.FactGraphServiceScenarioTest.java</exclude>
</excludes>
</configuration>
<executions>
<execution>
<goals>
<goal>test</goal>
</goals>
<!-- include end-to-end tests separately in the `integration-test` lifecycle phase -->
<phase>integration-test</phase>
<configuration>
<excludes>
<exclude>none</exclude>
</excludes>
<includes>
<include>**/*EndToEndTest</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<profiles>
<profile>development</profile>
<profile>debug</profile>
</profiles>
</configuration>
</plugin>
</plugins>
</build>
</project>

30
direct-file/backend/src/main/java/gov/irs/directfile/api/BackendApplication.java Normal file
View file

@ -0,0 +1,30 @@
package gov.irs.directfile.api;
import io.swagger.v3.oas.annotations.OpenAPIDefinition;
import io.swagger.v3.oas.annotations.info.Info;
import io.swagger.v3.oas.annotations.servers.Server;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.scheduling.annotation.EnableScheduling;
import gov.irs.directfile.api.config.*;
@SpringBootApplication
@EnableScheduling
@EnableConfigurationProperties({
StateApiEndpointProperties.class,
StateApiFeatureFlagProperties.class,
StatusEndpointProperties.class,
SubmitEndpointProperties.class,
})
@OpenAPIDefinition(
info = @Info(title = "Direct File API", description = "The Direct File API", version = "1.0.1"),
servers = {
@Server(url = "http://localhost:8080${server.servlet.context-path}", description = "Local development"),
})
public class BackendApplication {
public static void main(String[] args) {
SpringApplication.run(BackendApplication.class, args);
}
}

33
direct-file/backend/src/main/java/gov/irs/directfile/api/SessionController.java Normal file
View file

@ -0,0 +1,33 @@
package gov.irs.directfile.api;
import io.swagger.v3.oas.annotations.Operation;
import jakarta.servlet.http.HttpServletRequest;
import lombok.AllArgsConstructor;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import gov.irs.directfile.api.audit.Auditable;
import gov.irs.directfile.api.events.EventId;
import gov.irs.directfile.api.user.UserService;
@RestController
@RequestMapping("${direct-file.api-version}/session")
@AllArgsConstructor
public class SessionController {
private final UserService userService;
@Operation()
@Auditable(event = EventId.KEEP_ALIVE)
@GetMapping("/keep-alive")
public ResponseEntity<Void> keepAlive(HttpServletRequest request) {
userService.getCurrentUserInfo();
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
return new ResponseEntity<>(headers, HttpStatus.OK);
}
}

30
direct-file/backend/src/main/java/gov/irs/directfile/api/audit/AuditEventContextHolder.java Normal file
View file

@ -0,0 +1,30 @@
package gov.irs.directfile.api.audit;
import java.util.HashMap;
import java.util.Map;
import org.springframework.stereotype.Component;
import org.springframework.web.context.annotation.RequestScope;
@Component
@RequestScope
public class AuditEventContextHolder {
private final Map<String, Object> auditEventProperties = new HashMap<>();
private final Map<String, Object> auditEventDetailProperties = new HashMap<>();
public Map<String, Object> getEventContextProperties() {
Map<String, Object> outputProperties = new HashMap<>(auditEventProperties);
if (!auditEventDetailProperties.isEmpty()) {
outputProperties.put(AuditLogElement.DETAIL.toString(), auditEventDetailProperties);
}
return outputProperties;
}
public void addValueToEventMap(AuditLogElement key, Object value) {
auditEventProperties.put(key.toString(), value);
}
public void addValueToEventDetailMap(AuditLogElement.DetailElement key, String value) {
auditEventDetailProperties.put(key.toString(), value);
}
}

46
direct-file/backend/src/main/java/gov/irs/directfile/api/audit/AuditLogElement.java Normal file
View file

@ -0,0 +1,46 @@
package gov.irs.directfile.api.audit;
import org.apache.commons.text.CaseUtils;
public enum AuditLogElement {
CYBER_ONLY,
DETAIL,
EMAIL,
EVENT_ERROR_MESSAGE,
EVENT_ID,
EVENT_STATUS,
EVENT_TYPE,
GOOGLE_ANALYTICS_ID,
MEF_SUBMISSION_ID,
XXX_CODE,
REMOTE_ADDRESS,
REQUEST_METHOD,
REQUEST_URI,
RESPONSE_STATUS_CODE,
SADI_TID_HEADER,
SADI_USER_UUID,
STATE_ID,
TAX_PERIOD,
TAX_RETURN_ID,
TIN_TYPE,
TIMESTAMP,
USER_TIN,
USER_TIN_TYPE,
USER_TYPE,
DATA_IMPORT_BEHAVIOR;
@Override
public String toString() {
return CaseUtils.toCamelCase(super.toString(), false, '_');
}
public enum DetailElement {
STATE_ACCOUNT_ID,
MESSAGE;
@Override
public String toString() {
return CaseUtils.toCamelCase(super.toString(), false, '_');
}
}
}

49
direct-file/backend/src/main/java/gov/irs/directfile/api/audit/AuditService.java Normal file
View file

@ -0,0 +1,49 @@
package gov.irs.directfile.api.audit;
import lombok.AllArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.slf4j.MDC;
import org.slf4j.spi.LoggingEventBuilder;
import org.springframework.stereotype.Service;
import gov.irs.directfile.api.events.Event;
import gov.irs.directfile.api.events.EventStatus;
@Service
@AllArgsConstructor
@Slf4j
public class AuditService {
AuditEventContextHolder auditEventContextHolder;
public void addAuditPropertiesToMDC(final Event event) {
MDC.put(AuditLogElement.EVENT_STATUS.toString(), event.getEventStatus().toString());
MDC.put(AuditLogElement.EVENT_ID.toString(), event.getEventId().toString());
if (event.getEventPrincipal().getUserType() != null) {
MDC.put(
AuditLogElement.USER_TYPE.toString(),
event.getEventPrincipal().getUserType().toString());
}
if (event.getEventErrorMessage() != null) {
MDC.put(AuditLogElement.EVENT_ERROR_MESSAGE.toString(), event.getEventErrorMessage());
}
}
public void addEventProperty(AuditLogElement property, Object value) {
if (value != null) {
auditEventContextHolder.addValueToEventMap(property, value);
}
}
// keeping this package-private, there is currently no need to call it from outside this package
void performLog() {
LoggingEventBuilder builder =
EventStatus.SUCCESS.toString().equals(MDC.get(AuditLogElement.EVENT_STATUS.toString()))
? log.atInfo()
: log.atError();
// Add event-specific elements
auditEventContextHolder.getEventContextProperties().forEach(builder::addKeyValue);
builder.addKeyValue(AuditLogElement.CYBER_ONLY.toString(), true).log();
}
}

18
direct-file/backend/src/main/java/gov/irs/directfile/api/audit/Auditable.java Normal file
View file

@ -0,0 +1,18 @@
package gov.irs.directfile.api.audit;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import gov.irs.directfile.api.events.EventId;
import gov.irs.directfile.api.events.UserType;
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface Auditable {
EventId event();
UserType type() default UserType.SYS;
}

9
direct-file/backend/src/main/java/gov/irs/directfile/api/authentication/EnrollmentWindowException.java Normal file
View file

@ -0,0 +1,9 @@
package gov.irs.directfile.api.authentication;
import org.springframework.security.core.AuthenticationException;
public class EnrollmentWindowException extends AuthenticationException {
public EnrollmentWindowException(String message) {
super(message);
}
}

39
direct-file/backend/src/main/java/gov/irs/directfile/api/authentication/FakePIIService.java Normal file
View file

@ -0,0 +1,39 @@
package gov.irs.directfile.api.authentication;
import java.util.HashMap;
import java.util.Map;
import java.util.Set;
import java.util.UUID;
import lombok.AllArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Service;
@AllArgsConstructor
@Slf4j
@Service
public class FakePIIService implements PIIService {
private static final String PLACEHOLDER_ATTRIBUTE_VALUE = "FAKE_PII_PLACEHOLDER";
public static final String TIN = "123001234";
@Override
public Map<PIIAttribute, String> fetchAttributes(UUID userExternalId, Set<PIIAttribute> attributes) {
Map<PIIAttribute, String> responseMap = new HashMap<>();
for (PIIAttribute attribute : attributes) {
String attributeValue;
switch (attribute) {
case PIIAttribute.EMAILADDRESS -> attributeValue =
String.format("test-user+%s@directfile.test", userExternalId.toString());
case PIIAttribute.TIN -> {
attributeValue = TIN;
}
default -> attributeValue = PLACEHOLDER_ATTRIBUTE_VALUE;
}
responseMap.put(attribute, attributeValue);
}
return responseMap;
}
}

44
direct-file/backend/src/main/java/gov/irs/directfile/api/authentication/LocalUserDetailsCacheService.java Normal file
View file

@ -0,0 +1,44 @@
package gov.irs.directfile.api.authentication;
import java.time.Duration;
import java.util.Optional;
import java.util.UUID;
import com.google.common.cache.Cache;
import com.google.common.cache.CacheBuilder;
import org.springframework.security.core.userdetails.UserDetails;
public class LocalUserDetailsCacheService implements UserDetailsCacheService {
private final Cache<UUID, UserDetails> userDetailsCache;
public LocalUserDetailsCacheService(UserDetailsCacheProperties userDetailsCacheProperties) {
CacheBuilder<Object, Object> builder = CacheBuilder.newBuilder();
Long maxItems = userDetailsCacheProperties.maxItems();
if (maxItems != null) {
builder.maximumSize(maxItems);
}
Duration expireAfterWrite = userDetailsCacheProperties.expireAfterWrite();
if (expireAfterWrite != null) {
builder.expireAfterWrite(expireAfterWrite);
}
this.userDetailsCache = builder.build();
}
@Override
public Optional<UserDetails> get(UUID userExternalId) {
return Optional.ofNullable(userDetailsCache.getIfPresent(userExternalId));
}
@Override
public void put(UUID userExternalId, UserDetails userDetails) {
userDetailsCache.put(userExternalId, userDetails);
}
@Override
public void clear() {
userDetailsCache.invalidateAll();
}
}

3
direct-file/backend/src/main/java/gov/irs/directfile/api/authentication/NullAuthenticationException.java Normal file
View file

@ -0,0 +1,3 @@
package gov.irs.directfile.api.authentication;
public class NullAuthenticationException extends RuntimeException {}

21
direct-file/backend/src/main/java/gov/irs/directfile/api/authentication/PIIAttribute.java Normal file
View file

@ -0,0 +1,21 @@
package gov.irs.directfile.api.authentication;
// PII data elements that can be retrieved from the SADI PII service
public enum PIIAttribute {
GIVENNAME,
SURNAME,
MIDDLENAME,
TIN,
DATEOFBIRTH,
EMAILADDRESS,
LANDLINENUMBER,
MOBILENUMBER,
MAILINGADDRESS,
STREETADDRESSLINE1,
STREETADDRESSLINE2,
CITY,
STATE,
ZIP,
COUNTRY,
IRSCREATEDATE,
}

13
direct-file/backend/src/main/java/gov/irs/directfile/api/authentication/PIIService.java Normal file
View file

@ -0,0 +1,13 @@
package gov.irs.directfile.api.authentication;
import java.util.Map;
import java.util.Set;
import java.util.UUID;
public interface PIIService {
default String fetchAttribute(UUID userExternalId, PIIAttribute attribute) {
return fetchAttributes(userExternalId, Set.of(attribute)).get(attribute);
}
Map<PIIAttribute, String> fetchAttributes(UUID userExternalId, Set<PIIAttribute> attributes);
}

15
direct-file/backend/src/main/java/gov/irs/directfile/api/authentication/PIIServiceException.java Normal file
View file

@ -0,0 +1,15 @@
package gov.irs.directfile.api.authentication;
public class PIIServiceException extends RuntimeException {
public PIIServiceException(Throwable cause) {
super(cause);
}
public PIIServiceException(String message) {
super(message);
}
public PIIServiceException(String message, Throwable cause) {
super(message, cause);
}
}

49
direct-file/backend/src/main/java/gov/irs/directfile/api/authentication/RemoteUserDetailsCacheService.java Normal file
View file

@ -0,0 +1,49 @@
package gov.irs.directfile.api.authentication;
import java.time.Duration;
import java.util.Optional;
import java.util.UUID;
import org.springframework.security.core.userdetails.UserDetails;
import gov.irs.directfile.api.cache.CacheService;
import gov.irs.directfile.api.config.RedisConfiguration;
public class RemoteUserDetailsCacheService implements UserDetailsCacheService {
private final CacheService cacheService;
private final UserDetailsCacheProperties userDetailsCacheProperties;
public RemoteUserDetailsCacheService(
CacheService cacheService, UserDetailsCacheProperties userDetailsCacheProperties) {
this.cacheService = cacheService;
this.userDetailsCacheProperties = userDetailsCacheProperties;
}
@Override
public Optional<UserDetails> get(UUID userExternalId) {
SMUserDetailsProperties properties = cacheService.get(
RedisConfiguration.USERS_CACHE_NAME, userExternalId.toString(), SMUserDetailsProperties.class);
if (properties == null) {
return Optional.empty();
}
return Optional.of(new SMUserDetailsPrincipal(properties));
}
@Override
public void put(UUID userExternalId, UserDetails userDetails) {
SMUserDetailsProperties properties = new SMUserDetailsProperties((SMUserDetailsPrincipal) userDetails);
Duration expireAfterWrite = userDetailsCacheProperties.expireAfterWrite();
if (expireAfterWrite != null) {
cacheService.set(
RedisConfiguration.USERS_CACHE_NAME, userExternalId.toString(), properties, expireAfterWrite);
} else {
cacheService.set(RedisConfiguration.USERS_CACHE_NAME, userExternalId.toString(), properties);
}
}
@Override
public void clear() {
cacheService.clearCache(RedisConfiguration.USERS_CACHE_NAME);
}
}

49
direct-file/backend/src/main/java/gov/irs/directfile/api/authentication/SMUserDetailsPrincipal.java Normal file
View file

@ -0,0 +1,49 @@
package gov.irs.directfile.api.authentication;
import java.util.Collection;
import java.util.UUID;
import org.springframework.security.core.GrantedAuthority;
import org.springframework.security.core.userdetails.UserDetails;
@SuppressWarnings("PMD.ReturnEmptyCollectionRatherThanNull")
public record SMUserDetailsPrincipal(UUID id, UUID externalId, String email, String tin) implements UserDetails {
public SMUserDetailsPrincipal(SMUserDetailsProperties properties) {
this(properties.id(), properties.externalId(), properties.email(), properties.tin());
}
@Override
public Collection<? extends GrantedAuthority> getAuthorities() {
return null;
}
@Override
public String getUsername() {
return email;
}
@Override
public String getPassword() {
return null;
}
@Override
public boolean isAccountNonExpired() {
return true;
}
@Override
public boolean isAccountNonLocked() {
return true;
}
@Override
public boolean isCredentialsNonExpired() {
return true;
}
@Override
public boolean isEnabled() {
return true;
}
}

9
direct-file/backend/src/main/java/gov/irs/directfile/api/authentication/SMUserDetailsProperties.java Normal file
View file

@ -0,0 +1,9 @@
package gov.irs.directfile.api.authentication;
import java.util.UUID;
public record SMUserDetailsProperties(UUID id, UUID externalId, String email, String tin) {
public SMUserDetailsProperties(SMUserDetailsPrincipal principal) {
this(principal.id(), principal.externalId(), principal.email(), principal.tin());
}
}

8
direct-file/backend/src/main/java/gov/irs/directfile/api/authentication/UserDetailsCacheProperties.java Normal file
View file

@ -0,0 +1,8 @@
package gov.irs.directfile.api.authentication;
import java.time.Duration;
import org.springframework.boot.context.properties.ConfigurationProperties;
@ConfigurationProperties("direct-file.user-details-cache")
public record UserDetailsCacheProperties(Long maxItems, Duration expireAfterWrite) {}

14
direct-file/backend/src/main/java/gov/irs/directfile/api/authentication/UserDetailsCacheService.java Normal file
View file

@ -0,0 +1,14 @@
package gov.irs.directfile.api.authentication;
import java.util.Optional;
import java.util.UUID;
import org.springframework.security.core.userdetails.UserDetails;
public interface UserDetailsCacheService {
Optional<UserDetails> get(UUID userExternalId);
void put(UUID userExternalId, UserDetails userDetails);
void clear();
}

79
direct-file/backend/src/main/java/gov/irs/directfile/api/authorization/EmailAllowlistFeatureService.java Normal file
View file

@ -0,0 +1,79 @@
package gov.irs.directfile.api.authorization;
import java.nio.charset.StandardCharsets;
import java.util.Arrays;
import java.util.Base64;
import java.util.Collections;
import java.util.Set;
import java.util.stream.Collectors;
import lombok.Getter;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.bouncycastle.crypto.digests.SHA256Digest;
import org.bouncycastle.crypto.macs.HMac;
import org.bouncycastle.crypto.params.KeyParameter;
import org.springframework.stereotype.Service;
import gov.irs.directfile.api.authorization.config.FeatureFlagConfigurationProperties;
import gov.irs.directfile.api.featureflags.FeatureFlagService;
@Slf4j
@Service
public class EmailAllowlistFeatureService {
private final FeatureFlagService featureFlagService;
@Getter
private final boolean allowlistEnabled;
private Set<String> allowlist;
private final String allowListObject;
private final byte[] hexKey;
public EmailAllowlistFeatureService(
FeatureFlagConfigurationProperties configProps, FeatureFlagService featureFlagService) {
this.featureFlagService = featureFlagService;
this.allowlistEnabled = configProps.getAllowlist().enabled();
this.allowListObject = configProps.getAllowlist().objectKey();
this.hexKey = null;
}
// determines whether the identity provider-supplied email address is on our allowlist
public boolean emailOnAllowlist(String email) {
if (allowlistEnabled) {
loadAllowlist(); // trigger cache reload if needed
String base64Mac = emailMac(email);
return allowlist.contains(base64Mac);
}
// allowlist disabled
log.info("Allowlist is disabled, so emailOnAllowlist is false");
return false;
}
private String emailMac(String email) {
HMac hMac = new HMac(new SHA256Digest());
hMac.init(new KeyParameter(hexKey));
byte[] in = StringUtils.lowerCase(email).getBytes(StandardCharsets.UTF_8);
hMac.update(in, 0, in.length);
byte[] hMacOut = new byte[hMac.getMacSize()];
hMac.doFinal(hMacOut, 0);
return Base64.getEncoder().encodeToString(hMacOut);
}
private void loadAllowlist() {
if (!allowlistEnabled) {
return;
}
try {
this.allowlist = Arrays.stream(featureFlagService
.getFeatureObjectAsString(allowListObject)
.split("\n"))
.collect(Collectors.toSet());
log.info("Allowlist checked, total items: {}", allowlist.size());
} catch (Exception e) {
log.error("Error during allowlist retrieval: {}", e.getMessage());
this.allowlist = Collections.emptySet();
}
}
}

121
direct-file/backend/src/main/java/gov/irs/directfile/api/authorization/OpenEnrollmentFeatureService.java Normal file
View file

@ -0,0 +1,121 @@
package gov.irs.directfile.api.authorization;
import lombok.Getter;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Service;
import gov.irs.directfile.api.featureflags.FeatureFlagService;
import gov.irs.directfile.api.user.UserRepository;
@Slf4j
@Service
public class OpenEnrollmentFeatureService {
private final UserRepository userRepo;
private final FeatureFlagService featureFlagService;
@Getter
private final boolean openEnrollmentFeatureEnabled;
// This field is used to determine the current state of enrollment (open or closed).
// We read the set value for this in our feature-flags.json
// It's set to false by default, so that on startup the enrollment window is closed
// until we pick up feature-flags.json configuration.
private boolean newUsersAllowed = false;
// We read the set value for this in our feature-flags.json
private int maxUsersTarget;
private int currentUserCount;
public OpenEnrollmentFeatureService(UserRepository userRepo, FeatureFlagService featureFlagService) {
this.userRepo = userRepo;
this.featureFlagService = featureFlagService;
this.openEnrollmentFeatureEnabled = true;
}
/*In PROD, this schedule depends on the cron configuration to enable or disable it.
* To enable it, the cron value should be "".
* To disable it, the cron value should be "-".
* fixedDelayMilliseconds is the actual schedule frequency
* When the schedule is enabled, fixedDelayMilliseconds should be set to the desired fixed delay.
* When the schedule is disabled, fixedDelayMilliseconds should be blank */
protected void loadOpenEnrollmentConfig() {
if (!openEnrollmentFeatureEnabled) {
log.warn("Open enrollment feature is disabled, but the scheduled poller is running. "
+ "To shut off the poller, update the cron configuration and fixedDelayMilliseconds in the applicable application.yaml. "
+ "See comments in OpenEnrollmentFeatureService.java for more information.");
return;
}
try {
boolean newUsersAllowedFeatureFlag = true;
int maxUsersTargetConfig = 200000000;
// check to see if we should change our open enrollment state
if (this.newUsersAllowed != newUsersAllowedFeatureFlag) {
if (newUsersAllowedFeatureFlag) {
// if a new enrollment window is starting, set it up
startOpenEnrollment(maxUsersTargetConfig);
} else {
// if an active enrollment window is ending, reset local config
endOpenEnrollment();
}
} else if (this.newUsersAllowed) {
this.maxUsersTarget = maxUsersTargetConfig;
checkCurrentUserCount();
}
} catch (Exception e) {
log.error("Error during open enrollment configuration retrieval: {}", e.getMessage());
this.newUsersAllowed = false;
}
}
public boolean newUsersAllowed() {
if (!openEnrollmentFeatureEnabled) {
// if the entire feature is not enabled
// (e.g., this is the development environment),
// revert to default behavior where we allow new users
log.info("Open enrollment feature disabled, all users allowed.");
return true;
}
return newUsersAllowed && !maxUserCountReached();
}
private void checkCurrentUserCount() {
if (maxUserCountReached()) {
log.info(
"Reached max user count for the current open enrollment window. Total current users: {} with maximum: {}",
currentUserCount,
maxUsersTarget);
return;
}
updateCurrentUserCount();
log.info(
"Checked current user count. Total current users: {} with maximum: {}",
currentUserCount,
maxUsersTarget);
}
private boolean maxUserCountReached() {
return currentUserCount >= maxUsersTarget;
}
private void startOpenEnrollment(int maxUsersTarget) {
this.newUsersAllowed = true;
this.maxUsersTarget = maxUsersTarget;
updateCurrentUserCount();
log.info(
"Starting open enrollment window, max new users target: {}, current user count: {}",
maxUsersTarget,
currentUserCount);
}
private void endOpenEnrollment() {
this.newUsersAllowed = false;
this.maxUsersTarget = 0;
log.info("Ending open enrollment window");
}
private void updateCurrentUserCount() {
this.currentUserCount = userRepo.countByAccessGranted(true);
}
}

27
direct-file/backend/src/main/java/gov/irs/directfile/api/authorization/config/FeatureFlagConfigurationProperties.java Normal file
View file

@ -0,0 +1,27 @@
package gov.irs.directfile.api.authorization.config;
import jakarta.validation.constraints.NotNull;
import lombok.AllArgsConstructor;
import lombok.Getter;
import org.springframework.stereotype.Component;
@Getter
@Component
public class FeatureFlagConfigurationProperties {
@NotNull private final Allowlist allowlist;
@NotNull private final OpenEnrollment openEnrollment;
public FeatureFlagConfigurationProperties() {
this.allowlist = new Allowlist(false, "key", "allowlist.csv");
this.openEnrollment = new OpenEnrollment(true);
}
@Getter
@AllArgsConstructor
public static class OpenEnrollment {
@NotNull private final boolean enabled;
}
public record Allowlist(@NotNull boolean enabled, @NotNull String hexKey, @NotNull String objectKey) {}
}

57
direct-file/backend/src/main/java/gov/irs/directfile/api/authorization/config/StorageConfiguration.java Normal file
View file

@ -0,0 +1,57 @@
package gov.irs.directfile.api.authorization.config;
import java.net.URI;
import lombok.AllArgsConstructor;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.sts.StsClient;
import software.amazon.awssdk.services.sts.auth.StsAssumeRoleCredentialsProvider;
import software.amazon.awssdk.services.sts.model.AssumeRoleRequest;
import gov.irs.directfile.api.config.S3ConfigurationProperties;
@Configuration
@EnableConfigurationProperties(S3ConfigurationProperties.class)
@AllArgsConstructor
public class StorageConfiguration {
private final AwsCredentialsProvider awsCredentialsProvider;
@Profile("aws")
@Bean("s3WithoutEncryption")
S3Client s3Client(S3ConfigurationProperties s3ConfigurationProperties) {
return S3Client.builder()
.region(Region.of(s3ConfigurationProperties.getRegion()))
.credentialsProvider(StsAssumeRoleCredentialsProvider.builder()
.stsClient(StsClient.builder()
.region(Region.of(s3ConfigurationProperties.getRegion()))
.credentialsProvider(awsCredentialsProvider)
.build())
.refreshRequest(AssumeRoleRequest.builder()
.roleArn(s3ConfigurationProperties.getS3().getAssumeRoleArn())
.roleSessionName(
s3ConfigurationProperties.getS3().getAssumeRoleSessionName())
.durationSeconds(
s3ConfigurationProperties.getS3().getAssumeRoleDurationSeconds())
.build())
.build())
.endpointOverride(URI.create(s3ConfigurationProperties.getS3().getEndpoint()))
.build();
}
@Profile("!aws")
@Bean("s3WithoutEncryption")
S3Client localS3Client(S3ConfigurationProperties s3ConfigurationProperties) {
return S3Client.builder()
.region(Region.of(s3ConfigurationProperties.getRegion()))
.credentialsProvider(awsCredentialsProvider)
.endpointOverride(URI.create(s3ConfigurationProperties.getS3().getEndpoint()))
.build();
}
}

95
direct-file/backend/src/main/java/gov/irs/directfile/api/cache/CacheService.java vendored Normal file
View file

@ -0,0 +1,95 @@
package gov.irs.directfile.api.cache;
import java.time.Duration;
import java.util.Set;
import com.fasterxml.jackson.databind.ObjectMapper;
import lombok.AllArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.stereotype.Service;
@Service
@Slf4j
@AllArgsConstructor
public class CacheService {
private RedisTemplate<String, String> redisTemplate;
private ObjectMapper objectMapper;
public static final String KEY_SEPARATOR = ":";
public static final String KEY_GLOB = "*";
private String makeCacheKeyString(String cacheName, String key) {
return String.join(KEY_SEPARATOR, cacheName, key);
}
public <V> void set(String cacheName, String key, V value) {
set(cacheName, key, value, null);
}
public <V> void set(String cacheName, String key, V value, Duration timeout) {
String remoteCacheKeyString = makeCacheKeyString(cacheName, key);
try {
String serializedValue = objectMapper.writeValueAsString(value);
if (timeout != null) {
log.info("Setting cache key {} with timeout {}", remoteCacheKeyString, timeout);
redisTemplate.opsForValue().set(remoteCacheKeyString, serializedValue, timeout);
} else {
log.info("Setting cache key {}", remoteCacheKeyString);
redisTemplate.opsForValue().set(remoteCacheKeyString, serializedValue);
}
} catch (Exception e) {
log.error(
"Unable to set data from Redis. {}: {} {} {}",
e.getClass(),
e.getMessage(),
e.getCause(),
e.getStackTrace());
}
}
public <V> V get(String cacheName, String key, Class<V> clazz) {
String remoteCacheKeyString = makeCacheKeyString(cacheName, key);
try {
log.info("Getting cache key {}", remoteCacheKeyString);
String serializedValue = redisTemplate.opsForValue().get(remoteCacheKeyString);
return (serializedValue != null) ? objectMapper.readValue(serializedValue, clazz) : null;
} catch (Exception e) {
log.error(
"Unable to fetch data from Redis. {}: {} {} {}",
e.getClass(),
e.getMessage(),
e.getCause(),
e.getStackTrace());
return null;
}
}
public void clearKey(String cacheName, String key) {
String remoteCacheKeyString = makeCacheKeyString(cacheName, key);
try {
log.info("Clearing cache key {}", remoteCacheKeyString);
redisTemplate.delete(remoteCacheKeyString);
} catch (Exception e) {
log.error("Unable to clear key from Redis. {}: {}", e.getClass(), e.getMessage());
}
}
public void clearCache(String cacheName) {
String keyPatternToDelete = makeCacheKeyString(cacheName, KEY_GLOB);
try {
log.info("Clearing cache pattern {}", keyPatternToDelete);
Set<String> keys = redisTemplate.keys(keyPatternToDelete);
if (keys != null) {
redisTemplate.delete(keys);
}
} catch (Exception e) {
log.error(
"Unable to clear data from Redis. {}: {} {} {}",
e.getClass(),
e.getMessage(),
e.getCause(),
e.getStackTrace());
}
}
}

74
direct-file/backend/src/main/java/gov/irs/directfile/api/config/AopConfiguration.java Normal file
View file

@ -0,0 +1,74 @@
package gov.irs.directfile.api.config;
import java.lang.reflect.Method;
import lombok.AllArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.aspectj.lang.JoinPoint;
import org.aspectj.lang.annotation.AfterReturning;
import org.aspectj.lang.annotation.AfterThrowing;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.aspectj.lang.reflect.MethodSignature;
import org.springframework.context.annotation.EnableAspectJAutoProxy;
import org.springframework.stereotype.Component;
import gov.irs.directfile.api.audit.AuditEventContextHolder;
import gov.irs.directfile.api.audit.AuditLogElement;
import gov.irs.directfile.api.audit.AuditService;
import gov.irs.directfile.api.audit.Auditable;
import gov.irs.directfile.api.events.Event;
import gov.irs.directfile.api.events.EventPrincipal;
import gov.irs.directfile.api.events.EventStatus;
import gov.irs.directfile.api.events.SystemEventPrincipal;
import gov.irs.directfile.api.events.UserType;
@EnableAspectJAutoProxy
@Aspect
@Component
@AllArgsConstructor
@Slf4j
@SuppressWarnings({"PMD.UnusedFormalParameter"})
public class AopConfiguration {
private final AuditService auditService;
private final AuditEventContextHolder auditEventContextHolder;
@Pointcut("@annotation(gov.irs.directfile.api.audit.Auditable)")
public void auditableMethods() {}
@AfterReturning("auditableMethods()")
public void logAfterAuditableMethod(JoinPoint jp) {
MethodSignature signature = (MethodSignature) jp.getSignature();
// annotations
Method method = signature.getMethod();
Auditable auditableAnnotation = method.getAnnotation(Auditable.class);
auditService.addAuditPropertiesToMDC(Event.builder()
.eventId(auditableAnnotation.event())
.eventStatus(EventStatus.SUCCESS)
.eventPrincipal(createEventPrincipal(auditableAnnotation.type()))
.build());
}
@AfterThrowing(value = "auditableMethods()", throwing = "ex")
public void logAfterAuditableMethodException(JoinPoint jp, Throwable ex) {
MethodSignature signature = (MethodSignature) jp.getSignature();
// annotations
Method method = signature.getMethod();
Auditable auditableAnnotation = method.getAnnotation(Auditable.class);
auditEventContextHolder.addValueToEventDetailMap(AuditLogElement.DetailElement.MESSAGE, ex.getMessage());
auditService.addAuditPropertiesToMDC(Event.builder()
.eventId(auditableAnnotation.event())
.eventStatus(EventStatus.FAILURE)
.eventPrincipal(createEventPrincipal(auditableAnnotation.type()))
.eventErrorMessage(ex.getClass().getName())
.build());
}
private EventPrincipal createEventPrincipal(UserType userType) {
return new SystemEventPrincipal();
}
}

14
direct-file/backend/src/main/java/gov/irs/directfile/api/config/AwsConfigurationProperties.java Normal file
View file

@ -0,0 +1,14 @@
package gov.irs.directfile.api.config;
import lombok.AllArgsConstructor;
import lombok.Getter;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.validation.annotation.Validated;
@Validated
@ConfigurationProperties("aws")
@Getter
@AllArgsConstructor
public class AwsConfigurationProperties {
private final String kmsEndpoint;
}

32
direct-file/backend/src/main/java/gov/irs/directfile/api/config/AwsCredentialsConfiguration.java Normal file
View file

@ -0,0 +1,32 @@
package gov.irs.directfile.api.config;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
@Configuration
@EnableConfigurationProperties(S3ConfigurationProperties.class)
public class AwsCredentialsConfiguration {
@Bean
@ConditionalOnProperty(
name = "direct-file.aws.default-credentials-provider-chain-enabled",
havingValue = "false",
matchIfMissing = true)
public AwsCredentialsProvider staticCredentialsProvider(S3ConfigurationProperties s3ConfigurationProperties) {
return StaticCredentialsProvider.create(AwsBasicCredentials.create(
s3ConfigurationProperties.getCredentials().getAccessKey(),
s3ConfigurationProperties.getCredentials().getSecretKey()));
}
@Bean
@ConditionalOnProperty(name = "direct-file.aws.default-credentials-provider-chain-enabled", havingValue = "true")
public AwsCredentialsProvider defaultAWSCredentialsProvider() {
return DefaultCredentialsProvider.create();
}
}

10
direct-file/backend/src/main/java/gov/irs/directfile/api/config/BeanProfiles.java Normal file
View file

@ -0,0 +1,10 @@
package gov.irs.directfile.api.config;
public class BeanProfiles {
public static final String CORS_PERMIT_ALL = "cors-permit-all";
public static final String DEFAULT_SECURITY = "default-security";
public static final String ENABLE_DEVELOPMENT_IDENTITY_SUPPLIER = "enable-development-identity-supplier";
public static final String ENABLE_REMOTE_CACHE = "enable-remote-cache";
public static final String ALLOW_AUTHENTICATION_WITHOUT_PII = "allow-authentication-without-pii";
public static final String FAKE_PII_SERVICE = "fake-pii-service";
}

20
direct-file/backend/src/main/java/gov/irs/directfile/api/config/ConfiguredPdfProperties.java Normal file
View file

@ -0,0 +1,20 @@
package gov.irs.directfile.api.config;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.Setter;
@Getter
@Setter
@AllArgsConstructor
public class ConfiguredPdfProperties {
private String name;
private String year;
private String languageCode;
private String location;
private String locationType;
private String configurationLocation;
private String configurationLocationType;
private boolean cacheInMemory;
private int[] pagesToInclude;
}

17
direct-file/backend/src/main/java/gov/irs/directfile/api/config/DataImportGatingConfigurationProperties.java Normal file
View file

@ -0,0 +1,17 @@
package gov.irs.directfile.api.config;
import jakarta.validation.constraints.NotNull;
import lombok.AllArgsConstructor;
import lombok.Getter;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.validation.annotation.Validated;
@Validated
@ConfigurationProperties(prefix = "direct-file.data-import-gating")
@Getter
@AllArgsConstructor
public class DataImportGatingConfigurationProperties {
@NotNull private final Allowlist allowlist;
public record Allowlist(@NotNull boolean enabled, @NotNull String hexKey, @NotNull String objectKey) {}
}

18
direct-file/backend/src/main/java/gov/irs/directfile/api/config/DataImportGatingS3Properties.java Normal file
View file

@ -0,0 +1,18 @@
package gov.irs.directfile.api.config;
import java.time.Duration;
import jakarta.validation.constraints.NotNull;
import lombok.AllArgsConstructor;
import lombok.Getter;
import org.springframework.boot.context.properties.ConfigurationProperties;
@ConfigurationProperties(prefix = "direct-file.aws.s3")
@Getter
@AllArgsConstructor
public class DataImportGatingS3Properties {
private final String environmentPrefix;
private final @NotNull String dataImportGatingBucket;
private final @NotNull String dataImportGatingObject;
private final @NotNull Duration dataImportGatingExpiration;
}

18
direct-file/backend/src/main/java/gov/irs/directfile/api/config/DevCorsSecurityConfiguration.java Normal file
View file

@ -0,0 +1,18 @@
package gov.irs.directfile.api.config;
import org.springframework.boot.autoconfigure.condition.ConditionalOnWebApplication;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;
import org.springframework.web.servlet.config.annotation.CorsRegistry;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurer;
@Profile(BeanProfiles.CORS_PERMIT_ALL)
@Configuration
@ConditionalOnWebApplication
public class DevCorsSecurityConfiguration implements WebMvcConfigurer {
@Override
public void addCorsMappings(CorsRegistry registry) {
registry.addMapping("/**").allowedMethods("*");
}
}

55
direct-file/backend/src/main/java/gov/irs/directfile/api/config/DevelopmentIdentitySupplier.java Normal file
View file

@ -0,0 +1,55 @@
package gov.irs.directfile.api.config;
import java.util.HashMap;
import java.util.Map;
import java.util.Set;
import java.util.UUID;
import lombok.extern.slf4j.Slf4j;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.context.properties.ConfigurationPropertiesScan;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;
import gov.irs.directfile.api.authentication.FakePIIService;
import gov.irs.directfile.api.authentication.PIIAttribute;
import gov.irs.directfile.api.config.identity.IdentityAttributes;
import gov.irs.directfile.api.config.identity.IdentitySupplier;
@Configuration
@Profile(BeanProfiles.ENABLE_DEVELOPMENT_IDENTITY_SUPPLIER)
@ConfigurationPropertiesScan
@Slf4j
public class DevelopmentIdentitySupplier {
static UUID internalId = UUID.fromString("11111111-1111-1111-1111-111111111111");
static UUID externalId = UUID.fromString("00000000-0000-0000-0000-000000000000");
private static final FakePIIService fakePIIService = new FakePIIService();
private static final Set<PIIAttribute> piiAttributesToGenerate =
Set.of(PIIAttribute.EMAILADDRESS, PIIAttribute.TIN);
public record DevelopmentUserAttributes(String email, String tin) {}
@ConfigurationProperties(prefix = "direct-file.dev-data.identity-supplier")
public record DevelopmentUserProperties(Map<UUID, DevelopmentUserAttributes> userMap) {}
private final Map<UUID, DevelopmentUserAttributes> externalIdToEmailMap;
public DevelopmentIdentitySupplier(DevelopmentUserProperties developmentUserProperties) {
log.info(
"Running with development identity supplier. Users will be loaded from \"direct-file.dev-data.identity-supplier.user-map\"");
externalIdToEmailMap =
developmentUserProperties != null ? developmentUserProperties.userMap() : new HashMap<>();
}
@Bean
public IdentitySupplier getIdentitySupplierDevelopment() {
return () -> {
Map<PIIAttribute, String> piiAttributes =
fakePIIService.fetchAttributes(externalId, piiAttributesToGenerate);
String email = piiAttributes.get(PIIAttribute.EMAILADDRESS);
String tin = piiAttributes.get(PIIAttribute.TIN);
return new IdentityAttributes(internalId, externalId, email, tin);
};
}
}

15
direct-file/backend/src/main/java/gov/irs/directfile/api/config/DirectFileConfigurationProperties.java Normal file
View file

@ -0,0 +1,15 @@
package gov.irs.directfile.api.config;
import jakarta.validation.constraints.NotNull;
import lombok.AllArgsConstructor;
import lombok.Getter;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.validation.annotation.Validated;
@Validated
@ConfigurationProperties(prefix = "direct-file")
@Getter
@AllArgsConstructor
public class DirectFileConfigurationProperties {
@NotNull private String apiVersion;
}

23
direct-file/backend/src/main/java/gov/irs/directfile/api/config/HibernateConfiguration.java Normal file
View file

@ -0,0 +1,23 @@
package gov.irs.directfile.api.config;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.hibernate.cfg.AvailableSettings;
import org.hibernate.type.format.jackson.JacksonJsonFormatMapper;
import org.springframework.boot.autoconfigure.orm.jpa.HibernatePropertiesCustomizer;
import org.springframework.context.annotation.Bean;
import org.springframework.stereotype.Component;
/*
* See: https://github.com/spring-projects/spring-boot/issues/33870
*
* This is to get hibernate to serialize json the same way spring does
*/
@Component
public class HibernateConfiguration {
@Bean
HibernatePropertiesCustomizer jsonFormatMapperCustomizer(ObjectMapper objectMapper) {
return (properties) ->
properties.put(AvailableSettings.JSON_FORMAT_MAPPER, new JacksonJsonFormatMapper(objectMapper));
}
}

44
direct-file/backend/src/main/java/gov/irs/directfile/api/config/IPAddressUtil.java Normal file
View file

@ -0,0 +1,44 @@
package gov.irs.directfile.api.config;
import java.util.Arrays;
import java.util.List;
import java.util.Optional;
import jakarta.servlet.http.HttpServletRequest;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
@SuppressWarnings(value = {"PMD.AvoidUsingHardCodedIP", "PMD.SignatureDeclareThrowsException"})
@Slf4j
public class IPAddressUtil {
public static String getClientIpAddress(HttpServletRequest request) throws Exception {
String trueClientIpHeaderValue = request.getHeader(RequestHeaderNames.TRUE_CLIENT_IP);
if (StringUtils.isNotBlank(trueClientIpHeaderValue)) {
return trueClientIpHeaderValue.strip();
}
String addr = request.getHeader(RequestHeaderNames.X_FORWARDED_FOR);
if (StringUtils.isBlank(addr)) {
return request.getRemoteAddr();
}
String[] addrs = addr.split(",");
return getFirstIpAddress(addrs);
}
private static String getFirstIpAddress(String[] xffIpAddresses) throws Exception {
List<String> ipAddresses = Arrays.asList(xffIpAddresses);
Optional<String> result = ipAddresses.stream()
.filter(ip -> ip != null && !ip.isEmpty())
.map(String::strip)
.findFirst();
if (result.isPresent()) {
return result.get();
} else {
throw new Exception("No IP address found.");
}
}
}

31
direct-file/backend/src/main/java/gov/irs/directfile/api/config/KmsClientConfiguration.java Normal file
View file

@ -0,0 +1,31 @@
package gov.irs.directfile.api.config;
import java.net.URI;
import lombok.AllArgsConstructor;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.kms.KmsClient;
@Configuration
@EnableConfigurationProperties({AwsConfigurationProperties.class, S3ConfigurationProperties.class})
@AllArgsConstructor
public class KmsClientConfiguration {
private final AwsConfigurationProperties awsConfigurationProperties;
private final S3ConfigurationProperties s3ConfigurationProperties;
private final AwsCredentialsProvider awsCredentialsProvider;
@Bean
@Profile("aws")
public KmsClient regionalKmsClient() {
return KmsClient.builder()
.region(Region.of(s3ConfigurationProperties.getRegion()))
.credentialsProvider(awsCredentialsProvider)
.endpointOverride(URI.create(awsConfigurationProperties.getKmsEndpoint()))
.build();
}
}

16
direct-file/backend/src/main/java/gov/irs/directfile/api/config/LocalEncryptionConfigurationProperties.java Normal file
View file

@ -0,0 +1,16 @@
package gov.irs.directfile.api.config;
import jakarta.validation.constraints.NotBlank;
import lombok.AllArgsConstructor;
import lombok.Getter;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.validation.annotation.Validated;
@Validated
@ConfigurationProperties("direct-file.local-encryption")
@Getter
@AllArgsConstructor
public class LocalEncryptionConfigurationProperties {
@NotBlank
private final String localWrappingKey;
}

59
direct-file/backend/src/main/java/gov/irs/directfile/api/config/MessageQueueConfigurationProperties.java Normal file
View file

@ -0,0 +1,59 @@
package gov.irs.directfile.api.config;
import jakarta.validation.constraints.NotBlank;
import lombok.AllArgsConstructor;
import lombok.Getter;
import org.hibernate.validator.constraints.URL;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.validation.annotation.Validated;
@Validated
// ConfigurationProperties expects prefix to be in lowercase throws exception if "direct-file.aws.messageQueue" is used
// for example
@ConfigurationProperties("direct-file.aws.messagequeue")
@Getter
@AllArgsConstructor
public class MessageQueueConfigurationProperties {
@NotBlank
@URL
private final String endpoint;
private final boolean sqsMessageSendingEnabled;
@NotBlank
private final String dispatchQueue;
@NotBlank
private final String dlqStatusChangeQueue;
@NotBlank
private final String dlqSubmissionConfirmationQueue;
@NotBlank
private final String dlqS3NotificationEventQueue;
@NotBlank
private final String sendEmailQueue;
@NotBlank
private final String statusChangeQueue;
@NotBlank
private final String submissionConfirmationQueue;
@NotBlank
private final String s3NotificationEventQueue;
@NotBlank
private final String dataImportRequestQueue;
@NotBlank
private final String dlqDataImportRequestQueue;
@NotBlank
private final String dataImportResultQueue;
@NotBlank
private final String dlqDataImportResultQueue;
}

64
direct-file/backend/src/main/java/gov/irs/directfile/api/config/OpenApiConfiguration.java Normal file
View file

@ -0,0 +1,64 @@
package gov.irs.directfile.api.config;
import java.util.List;
import org.springdoc.core.customizers.OpenApiCustomizer;
import org.springdoc.core.models.GroupedOpenApi;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
@EnableConfigurationProperties(DirectFileConfigurationProperties.class)
public class OpenApiConfiguration {
private final String apiVersion;
private final List<String> externalPaths =
List.of("/taxreturns/**", "/users/**", "/state-api/state-profile", "/state-api/authorization-code");
private final List<String> internalPaths = List.of("/state-api/state-exported-facts/**", "/state-api/status/**");
private final List<String> demoPaths = List.of("/taxreturns-demo/**", "/debug/**", "/loaders/**");
public OpenApiConfiguration(DirectFileConfigurationProperties configProps) {
apiVersion = configProps.getApiVersion();
}
@Bean
public GroupedOpenApi externalEndpointsOpenApiGroup() {
return GroupedOpenApi.builder()
.group("external")
.displayName("external endpoints")
.pathsToMatch(getPathsWithApiVersionPrefix(externalPaths))
.build();
}
@Bean
public GroupedOpenApi internalEndpointsOpenApiGroup() {
return GroupedOpenApi.builder()
.group("internal")
.displayName("internal endpoints")
.pathsToMatch(getPathsWithApiVersionPrefix(internalPaths))
.addOpenApiCustomizer(removeSecuritySchemes())
.build();
}
@Bean
public GroupedOpenApi developmentOnlyOpenApiGroup() {
return GroupedOpenApi.builder()
.group("development")
.displayName("development endpoints")
.pathsToMatch(getPathsWithApiVersionPrefix(demoPaths))
.build();
}
@Bean
public OpenApiCustomizer removeSecuritySchemes() {
return openApi -> {
// Clear all security requirements
openApi.getComponents().setSecuritySchemes(null);
openApi.getSecurity().clear();
};
}
private String[] getPathsWithApiVersionPrefix(List<String> paths) {
return paths.stream().map(path -> "/" + apiVersion + path).toList().toArray(String[]::new);
}
}

21
direct-file/backend/src/main/java/gov/irs/directfile/api/config/PdfServiceProperties.java Normal file
View file

@ -0,0 +1,21 @@
package gov.irs.directfile.api.config;
import java.util.List;
import edu.umd.cs.findbugs.annotations.SuppressFBWarnings;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.Setter;
import org.springframework.boot.context.properties.ConfigurationProperties;
@Getter
@Setter
@AllArgsConstructor
@ConfigurationProperties(prefix = "direct-file.pdfs")
@SuppressFBWarnings(value = "EI_EXPOSE_REP", justification = "Initial Spotbugs Setup")
public class PdfServiceProperties {
private List<ConfiguredPdfProperties> configuredPdfs;
private String outputLocation;
private String outputLocationType;
private boolean useDocumentStorageForPilotYear;
}

30
direct-file/backend/src/main/java/gov/irs/directfile/api/config/RedisConfiguration.java Normal file
View file

@ -0,0 +1,30 @@
package gov.irs.directfile.api.config;
import lombok.extern.slf4j.Slf4j;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.serializer.JdkSerializationRedisSerializer;
import org.springframework.data.redis.serializer.StringRedisSerializer;
@Configuration
@Slf4j
public class RedisConfiguration {
public static final String FEATURE_FLAG_CACHE_NAME = "feature-flags";
public static final String STATUS_CACHE_NAME = "status";
public static final String USERS_CACHE_NAME = "users";
public static final String DATA_IMPORT_GATING_CACHE_NAME = "data-import-gating";
@Bean
RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory connectionFactory) {
log.info("RedisConnectionFactory: {}", connectionFactory);
RedisTemplate<String, Object> redisTemplate = new RedisTemplate<>();
redisTemplate.setConnectionFactory(connectionFactory);
redisTemplate.setKeySerializer(new StringRedisSerializer());
// Note: may need to change the JdkSerializationRedisSerializer when implementing encryption
redisTemplate.setValueSerializer(new JdkSerializationRedisSerializer());
return redisTemplate;
}
}

10
direct-file/backend/src/main/java/gov/irs/directfile/api/config/RequestHeaderNames.java Normal file
View file

@ -0,0 +1,10 @@
package gov.irs.directfile.api.config;
public class RequestHeaderNames {
public static final String PREAUTHENTICATED_AUTHENTICATION_HEADER_NAME = "SM_UNIVERSALID";
public static final String COOKIE = "Cookie";
public static final String CORRELATION_ID = "Correlation-ID";
public static final String CREDENTIAL_SERVICE_PROVIDER_TID_HEADER = "TID";
public static final String TRUE_CLIENT_IP = "True-Client-IP";
public static final String X_FORWARDED_FOR = "X-Forwarded-For";
}

59
direct-file/backend/src/main/java/gov/irs/directfile/api/config/S3ConfigurationProperties.java Normal file
View file

@ -0,0 +1,59 @@
package gov.irs.directfile.api.config;
import jakarta.validation.constraints.Min;
import jakarta.validation.constraints.NotBlank;
import jakarta.validation.constraints.NotNull;
import lombok.AllArgsConstructor;
import lombok.Getter;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.validation.annotation.Validated;
@Validated
@ConfigurationProperties("direct-file.aws")
@Getter
@AllArgsConstructor
public class S3ConfigurationProperties {
@NotBlank
private final String region;
@NotNull private final Credentials credentials;
@NotNull private final S3 s3;
@Getter
@AllArgsConstructor
public static class Credentials {
@NotBlank
private final String accessKey;
@NotBlank
private final String secretKey;
}
@Getter
@AllArgsConstructor
public static class S3 {
@NotBlank
private final String endpoint;
@NotBlank
private final String assumeRoleArn;
@NotNull @Min(value = 0)
private final int assumeRoleDurationSeconds;
@NotBlank
private final String assumeRoleSessionName;
private final String kmsWrappingKeyArn;
@NotBlank
private final String bucket;
@NotBlank
private final String operationsJobsBucket;
private final String environmentPrefix;
}
}

68
direct-file/backend/src/main/java/gov/irs/directfile/api/config/S3StorageConfig.java Normal file
View file

@ -0,0 +1,68 @@
package gov.irs.directfile.api.config;
import java.net.URI;
import java.util.Base64;
import javax.crypto.SecretKey;
import javax.crypto.spec.SecretKeySpec;
import lombok.AllArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.context.annotation.Profile;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3AsyncClient;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.encryption.s3.S3EncryptionClient;
@Slf4j
@Configuration
@AllArgsConstructor
public class S3StorageConfig {
private final AwsCredentialsProvider awsCredentialsProvider;
@Profile("!aws")
@EnableConfigurationProperties({S3ConfigurationProperties.class, LocalEncryptionConfigurationProperties.class})
class Local {
@Bean
@Primary
S3EncryptionClient s3LocalEncryptionClient(
S3ConfigurationProperties s3ConfigurationProperties,
LocalEncryptionConfigurationProperties localEncryptionConfigurationProperties) {
log.warn("S3: Using local encryption without AWS KMS. Not appropriate for deployed environments!");
return S3EncryptionClient.builder()
.wrappedClient(staticCredentialClient(s3ConfigurationProperties))
.wrappedAsyncClient(asyncStaticCredentialClient(s3ConfigurationProperties))
.aesKey(getLocalAesWrappingKey(localEncryptionConfigurationProperties))
.build();
}
private S3Client staticCredentialClient(S3ConfigurationProperties s3ConfigurationProperties) {
return S3Client.builder()
.region(Region.of(s3ConfigurationProperties.getRegion()))
.credentialsProvider(awsCredentialsProvider)
.endpointOverride(
URI.create(s3ConfigurationProperties.getS3().getEndpoint()))
.build();
}
private S3AsyncClient asyncStaticCredentialClient(S3ConfigurationProperties s3ConfigurationProperties) {
return S3AsyncClient.builder()
.region(Region.of(s3ConfigurationProperties.getRegion()))
.credentialsProvider(awsCredentialsProvider)
.endpointOverride(
URI.create(s3ConfigurationProperties.getS3().getEndpoint()))
.build();
}
private SecretKey getLocalAesWrappingKey(
LocalEncryptionConfigurationProperties localEncryptionConfigurationProperties) {
return new SecretKeySpec(
Base64.getDecoder().decode(localEncryptionConfigurationProperties.getLocalWrappingKey()), "AES");
}
}
}

66
direct-file/backend/src/main/java/gov/irs/directfile/api/config/SecurityConfiguration.java Normal file
View file

@ -0,0 +1,66 @@
package gov.irs.directfile.api.config;
import lombok.extern.slf4j.Slf4j;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;
import org.springframework.core.annotation.Order;
import org.springframework.security.config.Customizer;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configurers.AbstractHttpConfigurer;
import org.springframework.security.config.http.SessionCreationPolicy;
import org.springframework.security.web.SecurityFilterChain;
import gov.irs.directfile.api.authentication.*;
import gov.irs.directfile.api.cache.CacheService;
@Configuration
@Profile(BeanProfiles.DEFAULT_SECURITY)
@EnableConfigurationProperties({UserDetailsCacheProperties.class})
@Slf4j
public class SecurityConfiguration {
@Bean
@Profile(BeanProfiles.FAKE_PII_SERVICE)
public PIIService fakePiiService() {
return new FakePIIService();
}
@Bean
@Profile("!" + BeanProfiles.ENABLE_REMOTE_CACHE)
public UserDetailsCacheService localUserDetailsCacheService(UserDetailsCacheProperties userDetailsCacheProperties) {
return new LocalUserDetailsCacheService(userDetailsCacheProperties);
}
@Bean
@Profile(BeanProfiles.ENABLE_REMOTE_CACHE) // use remote cache for local development
public UserDetailsCacheService remoteUserDetailsCacheService(
CacheService cacheService, UserDetailsCacheProperties userDetailsCacheProperties) {
return new RemoteUserDetailsCacheService(cacheService, userDetailsCacheProperties);
}
@Bean
@Order(1)
public SecurityFilterChain fc(HttpSecurity http) {
// This chain handles all paths that **do not** require authentication
log.info("Adding SecurityFilterChain: anonymousFilterChain");
try {
http.csrf(AbstractHttpConfigurer::disable)
.cors(Customizer.withDefaults())
.securityMatchers(securityMatchers -> securityMatchers.requestMatchers("/**"))
.authorizeHttpRequests(authorize -> authorize.anyRequest().permitAll())
.sessionManagement(
sessionMgmt -> sessionMgmt.sessionCreationPolicy(SessionCreationPolicy.STATELESS))
.httpBasic(AbstractHttpConfigurer::disable)
.formLogin(AbstractHttpConfigurer::disable)
.logout(AbstractHttpConfigurer::disable);
return http.build();
} catch (Exception e) {
log.error("Anonymous HttpSecurity filter fails: {}", e.getMessage());
throw new RuntimeException("Anonymous HttpSecurity filter fails: " + e.getMessage(), e);
}
}
}

29
direct-file/backend/src/main/java/gov/irs/directfile/api/config/SnsClientConfiguration.java Normal file
View file

@ -0,0 +1,29 @@
package gov.irs.directfile.api.config;
import java.net.URI;
import lombok.AllArgsConstructor;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.sns.SnsClient;
@Configuration
@ConditionalOnProperty(value = "direct-file.aws.sns.submission-confirmation-publish-enabled", havingValue = "true")
@EnableConfigurationProperties(SnsConfigurationProperties.class)
@AllArgsConstructor
public class SnsClientConfiguration {
private final AwsCredentialsProvider awsCredentialsProvider;
@Bean
public SnsClient snsClient(SnsConfigurationProperties snsConfigurationProperties) {
return SnsClient.builder()
.region(Region.of(snsConfigurationProperties.getRegion()))
.credentialsProvider(awsCredentialsProvider)
.endpointOverride(URI.create(snsConfigurationProperties.getEndpoint()))
.build();
}
}

26
direct-file/backend/src/main/java/gov/irs/directfile/api/config/SnsConfigurationProperties.java Normal file
View file

@ -0,0 +1,26 @@
package gov.irs.directfile.api.config;
import jakarta.validation.constraints.NotBlank;
import lombok.AllArgsConstructor;
import lombok.Getter;
import org.hibernate.validator.constraints.URL;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.validation.annotation.Validated;
@Validated
@ConfigurationProperties("direct-file.aws.sns")
@Getter
@AllArgsConstructor
public class SnsConfigurationProperties {
@NotBlank
@URL
private final String endpoint;
@NotBlank
private final String submissionConfirmationTopicArn;
private final boolean submissionConfirmationPublishEnabled;
@NotBlank
private final String region;
}

49
direct-file/backend/src/main/java/gov/irs/directfile/api/config/SqsMessageQueueConfiguration.java Normal file
View file

@ -0,0 +1,49 @@
package gov.irs.directfile.api.config;
import java.net.URI;
import com.amazon.sqs.javamessaging.ProviderConfiguration;
import com.amazon.sqs.javamessaging.SQSConnectionFactory;
import jakarta.jms.Connection;
import jakarta.jms.JMSException;
import lombok.AllArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.sqs.SqsClient;
@Configuration
@EnableConfigurationProperties({S3ConfigurationProperties.class, MessageQueueConfigurationProperties.class})
@Slf4j
@AllArgsConstructor
public class SqsMessageQueueConfiguration {
private final AwsCredentialsProvider awsCredentialsProvider;
@Bean
SqsClient getSqs(
S3ConfigurationProperties s3ConfigurationProperties,
MessageQueueConfigurationProperties messageQueueConfigurationProperties) {
return SqsClient.builder()
.region(Region.of(s3ConfigurationProperties.getRegion()))
.credentialsProvider(awsCredentialsProvider)
.endpointOverride(URI.create(messageQueueConfigurationProperties.getEndpoint()))
.build();
}
@Bean
public SQSConnectionFactory sqsConnectionFactory(SqsClient sqsClient) {
return new SQSConnectionFactory(new ProviderConfiguration(), sqsClient);
}
@Bean
public Connection jmsConnection(SQSConnectionFactory sqsConnectionFactory) throws JMSException {
Connection connection = sqsConnectionFactory.createConnection();
connection.start(); // Start the connection to enable message delivery
log.info("CONNECTED TO SQS");
return connection;
}
}

14
direct-file/backend/src/main/java/gov/irs/directfile/api/config/StateApiEndpointProperties.java Normal file
View file

@ -0,0 +1,14 @@
package gov.irs.directfile.api.config;
import lombok.AllArgsConstructor;
import lombok.Getter;
import org.springframework.boot.context.properties.ConfigurationProperties;
@AllArgsConstructor
@Getter
@ConfigurationProperties(prefix = "direct-file.state-api")
public class StateApiEndpointProperties {
private String baseUrl;
private String v2AuthTokenPath;
private String version;
}

17
direct-file/backend/src/main/java/gov/irs/directfile/api/config/StateApiFeatureFlagProperties.java Normal file
View file

@ -0,0 +1,17 @@
package gov.irs.directfile.api.config;
import jakarta.validation.constraints.NotNull;
import lombok.AllArgsConstructor;
import lombok.Getter;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.validation.annotation.Validated;
@ConfigurationProperties(prefix = "direct-file.feature-flags.state-api")
@Getter
@AllArgsConstructor
@Validated
public class StateApiFeatureFlagProperties {
@NotNull private final ExportedFacts exportedFacts;
public record ExportedFacts(@NotNull boolean enabled) {}
}

29
direct-file/backend/src/main/java/gov/irs/directfile/api/config/StatusConfiguration.java Normal file
View file

@ -0,0 +1,29 @@
package gov.irs.directfile.api.config;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;
import gov.irs.directfile.api.cache.CacheService;
import gov.irs.directfile.api.taxreturn.LocalStatusResponseBodyCacheService;
import gov.irs.directfile.api.taxreturn.RemoteStatusResponseBodyCacheService;
import gov.irs.directfile.api.taxreturn.StatusResponseBodyCacheService;
@Configuration
@EnableConfigurationProperties(StatusResponseBodyCacheProperties.class)
public class StatusConfiguration {
@Bean
@Profile("!" + BeanProfiles.ENABLE_REMOTE_CACHE) // use in memory cache
public StatusResponseBodyCacheService localStatusResponseBodyCacheService(
StatusResponseBodyCacheProperties statusResponseBodyCacheProperties) {
return new LocalStatusResponseBodyCacheService(statusResponseBodyCacheProperties);
}
@Bean
@Profile(BeanProfiles.ENABLE_REMOTE_CACHE) // use remote cache for local development
public StatusResponseBodyCacheService remoteStatusResponseBodyCacheService(
CacheService cacheService, StatusResponseBodyCacheProperties statusResponseBodyCacheProperties) {
return new RemoteStatusResponseBodyCacheService(cacheService, statusResponseBodyCacheProperties);
}
}

22
direct-file/backend/src/main/java/gov/irs/directfile/api/config/StatusEndpointProperties.java Normal file
View file

@ -0,0 +1,22 @@
package gov.irs.directfile.api.config;
import jakarta.validation.constraints.NotBlank;
import lombok.AllArgsConstructor;
import lombok.Getter;
import org.hibernate.validator.constraints.URL;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.validation.annotation.Validated;
@AllArgsConstructor
@Getter
@ConfigurationProperties(prefix = "direct-file.status-endpoint")
@Validated
public class StatusEndpointProperties {
@NotBlank
@URL
private String statusEndpointURI;
@NotBlank
@URL
private String rejectionCodesEndpointURI;
}

8
direct-file/backend/src/main/java/gov/irs/directfile/api/config/StatusResponseBodyCacheProperties.java Normal file
View file

@ -0,0 +1,8 @@
package gov.irs.directfile.api.config;
import java.time.Duration;
import org.springframework.boot.context.properties.ConfigurationProperties;
@ConfigurationProperties("direct-file.status-response-body-cache")
public record StatusResponseBodyCacheProperties(Long maxItems, Duration expireAfterWrite) {}

12
direct-file/backend/src/main/java/gov/irs/directfile/api/config/SubmitEndpointProperties.java Normal file
View file

@ -0,0 +1,12 @@
package gov.irs.directfile.api.config;
import lombok.AllArgsConstructor;
import lombok.Getter;
import org.springframework.boot.context.properties.ConfigurationProperties;
@AllArgsConstructor
@Getter
@ConfigurationProperties(prefix = "direct-file.submit-endpoint")
public class SubmitEndpointProperties {
private String submitEndpointURI;
}

16
direct-file/backend/src/main/java/gov/irs/directfile/api/config/SystemClockConfig.java Normal file
View file

@ -0,0 +1,16 @@
package gov.irs.directfile.api.config;
import java.time.*;
import lombok.extern.slf4j.Slf4j;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
@Slf4j
public class SystemClockConfig {
@Bean
public Clock systemClock() {
return Clock.systemUTC();
}
}

5
direct-file/backend/src/main/java/gov/irs/directfile/api/config/identity/IdentityAttributes.java Normal file
View file

@ -0,0 +1,5 @@
package gov.irs.directfile.api.config.identity;
import java.util.UUID;
public record IdentityAttributes(UUID id, UUID externalId, String email, String tin) {}

5
direct-file/backend/src/main/java/gov/irs/directfile/api/config/identity/IdentitySupplier.java Normal file
View file

@ -0,0 +1,5 @@
package gov.irs.directfile.api.config.identity;
import java.util.function.Supplier;
public interface IdentitySupplier extends Supplier<IdentityAttributes> {}

20
direct-file/backend/src/main/java/gov/irs/directfile/api/dataimport/DataImportRepository.java Normal file
View file

@ -0,0 +1,20 @@
package gov.irs.directfile.api.dataimport;
import java.util.List;
import java.util.Optional;
import java.util.UUID;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Query;
import gov.irs.directfile.api.dataimport.model.PopulatedData;
public interface DataImportRepository extends JpaRepository<PopulatedData, UUID> {
@Query(
value =
"WITH descending as (SELECT * FROM populated_data WHERE taxreturn_id = :taxReturnId ORDER BY created_at DESC) SELECT DISTINCT ON (source) * from descending",
nativeQuery = true)
List<PopulatedData> findLatestSourcesByTaxReturnId(UUID taxReturnId);
Optional<PopulatedData> findByTaxReturnIdAndSource(UUID taxReturnId, String source);
}

12
direct-file/backend/src/main/java/gov/irs/directfile/api/dataimport/DataImportService.java Normal file
View file

@ -0,0 +1,12 @@
package gov.irs.directfile.api.dataimport;
import java.util.Date;
import java.util.UUID;
import gov.irs.directfile.api.dataimport.model.WrappedPopulatedData;
public interface DataImportService {
void sendPreFetchRequest(UUID taxReturnId, UUID userId, UUID externalId, String tin, int taxYear);
WrappedPopulatedData getPopulatedData(UUID taxReturnId, UUID userId, Date taxReturnCreatedAt);
}

99
direct-file/backend/src/main/java/gov/irs/directfile/api/dataimport/MockDataImportService.java Normal file
View file

@ -0,0 +1,99 @@
package gov.irs.directfile.api.dataimport;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.Date;
import java.util.HashMap;
import java.util.Map;
import java.util.UUID;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ObjectNode;
import lombok.extern.slf4j.Slf4j;
import org.springframework.context.annotation.Profile;
import org.springframework.core.io.Resource;
import org.springframework.core.io.ResourceLoader;
import org.springframework.core.io.support.ResourcePatternUtils;
import org.springframework.stereotype.Service;
import org.springframework.util.StringUtils;
import gov.irs.directfile.api.dataimport.model.WrappedPopulatedData;
@Slf4j
@Service
@Profile("mock")
public class MockDataImportService implements DataImportService {
protected static final String PATH = "classpath:dataimportservice/mocks/*.json";
private Map<String, WrappedPopulatedData> map = new HashMap<>();
private Map<String, Long> TIMEOUT_INCREMENTER_MAP = new HashMap<>();
long RETRY_TIMEOUT = 20000;
public MockDataImportService(ObjectMapper mapper, ResourceLoader loader) {
try {
for (Resource resource :
ResourcePatternUtils.getResourcePatternResolver(loader).getResources(PATH)) {
String fileName = resource.getFilename();
if (fileName != null) {
try {
String file = fileName.replace(".json", "");
WrappedPopulatedData data = mapper.readValue(
resource.getContentAsString(StandardCharsets.UTF_8), WrappedPopulatedData.class);
map.put(file, data);
TIMEOUT_INCREMENTER_MAP.put(file, data.getData().getTimeSinceCreation());
} catch (IOException e) {
log.error(
"failed to read: {} {} {}", resource.getFilename(), e.getMessage(), e.getStackTrace());
}
}
}
} catch (IOException e) {
log.error("failed to load mock data {} {}", e.getMessage(), e.getStackTrace());
}
}
@Override
public void sendPreFetchRequest(UUID taxReturnId, UUID userId, UUID externalId, String tin, int taxYear) {
log.info(
"Mock prefetch called for Tax Return: {}, User ID: {}, External ID: {}",
taxReturnId,
userId,
externalId.toString());
}
@Override
public WrappedPopulatedData getPopulatedData(UUID taxReturnId, UUID userId, Date taxReturnCreatedAt) {
log.warn("Unexpected call for Tax Return: {}; User ID: {}", taxReturnId, userId);
return WrappedPopulatedData.from(new ArrayList<>(), taxReturnCreatedAt);
}
public WrappedPopulatedData getPopulatedData(String key, String dateOfBirth) {
log.info("Mock get data called for: {}", key);
WrappedPopulatedData data =
map.containsKey(key) ? map.get(key) : WrappedPopulatedData.from(new ArrayList<>(), new Date());
if (StringUtils.hasText(dateOfBirth)) {
for (JsonNode json : data.getData().getAboutYouBasic().getPayload()) {
if (json.hasNonNull("dateOfBirth")) {
((ObjectNode) json).put("dateOfBirth", dateOfBirth);
}
}
}
long nextTimeSinceCreation = TIMEOUT_INCREMENTER_MAP.get(key) + 1000;
if (nextTimeSinceCreation < RETRY_TIMEOUT) {
TIMEOUT_INCREMENTER_MAP.put(key, nextTimeSinceCreation);
} else {
TIMEOUT_INCREMENTER_MAP.put(key, 1000L);
}
return new WrappedPopulatedData(new WrappedPopulatedData.Data(
data.getData().getAboutYouBasic(),
data.getData().getIpPin(),
data.getData().getW2s(),
data.getData().getF1099Ints(),
data.getData().getF1095a(),
nextTimeSinceCreation));
}
}

13
direct-file/backend/src/main/java/gov/irs/directfile/api/dataimport/exception/DataImportException.java Normal file
View file

@ -0,0 +1,13 @@
package gov.irs.directfile.api.dataimport.exception;
public class DataImportException extends RuntimeException {
public DataImportException() {}
public DataImportException(String message) {
super(message);
}
public DataImportException(String message, Throwable t) {
super(message, t);
}
}

9
direct-file/backend/src/main/java/gov/irs/directfile/api/dataimport/gating/DataImportBehavior.java Normal file
View file

@ -0,0 +1,9 @@
package gov.irs.directfile.api.dataimport.gating;
public enum DataImportBehavior {
DATA_IMPORT_ABOUT_YOU_BASIC,
DATA_IMPORT_ABOUT_YOU_BASIC_PLUS_IP_PIN,
DATA_IMPORT_ABOUT_YOU_BASIC_PLUS_IP_PIN_PLUS_W2,
DATA_IMPORT_ABOUT_YOU_BASIC_PLUS_IP_PIN_PLUS_W2_PLUS_1099_INT,
DATA_IMPORT_ABOUT_YOU_BASIC_PLUS_IP_PIN_PLUS_W2_PLUS_1099_INT_PLUS_1095_A;
}

30
direct-file/backend/src/main/java/gov/irs/directfile/api/dataimport/gating/DataImportGatingConfig.java Normal file
View file

@ -0,0 +1,30 @@
package gov.irs.directfile.api.dataimport.gating;
import java.time.ZonedDateTime;
import java.util.List;
import edu.umd.cs.findbugs.annotations.SuppressFBWarnings;
import lombok.Getter;
import lombok.Setter;
@Getter
@Setter
@SuppressFBWarnings(value = {"UUF_UNUSED_FIELD"})
public class DataImportGatingConfig {
private List<Percentage> percentages;
private List<Windowing> windowing;
@Getter
@Setter
public static class Percentage {
private String behavior;
private int percentage;
}
@Getter
@Setter
public static class Windowing {
private ZonedDateTime start;
private ZonedDateTime end;
}
}

144
direct-file/backend/src/main/java/gov/irs/directfile/api/dataimport/gating/DataImportGatingConfigService.java Normal file
View file

@ -0,0 +1,144 @@
package gov.irs.directfile.api.dataimport.gating;
import java.io.IOException;
import java.nio.charset.Charset;
import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.PropertyNamingStrategies;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.stereotype.Service;
import software.amazon.awssdk.core.ResponseBytes;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.GetObjectRequest;
import software.amazon.awssdk.services.s3.model.GetObjectResponse;
import gov.irs.directfile.api.cache.CacheService;
import gov.irs.directfile.api.config.DataImportGatingConfigurationProperties;
import gov.irs.directfile.api.config.DataImportGatingS3Properties;
import gov.irs.directfile.api.config.RedisConfiguration;
import gov.irs.directfile.api.dataimport.exception.DataImportException;
@Slf4j
@Service
@EnableConfigurationProperties({DataImportGatingS3Properties.class, DataImportGatingConfigurationProperties.class})
public class DataImportGatingConfigService {
private final S3Client s3Client;
private final DataImportGatingS3Properties gatingS3Config;
private final ObjectMapper deserializationObjectMapper;
private final CacheService cacheService;
public DataImportGatingConfigService(
@Qualifier("s3WithoutEncryption") S3Client s3Client,
DataImportGatingS3Properties dataImportGatingConfig,
CacheService cacheService) {
this.s3Client = s3Client;
this.gatingS3Config = dataImportGatingConfig;
this.cacheService = cacheService;
// Deserialize from kebab case as the properties appear in the data import behavior file
// Serialization is handled by our default object mapper in the controller (camel case)
ObjectMapper deserializationObjectMapper = new ObjectMapper();
deserializationObjectMapper.setPropertyNamingStrategy(new PropertyNamingStrategies.KebabCaseStrategy());
deserializationObjectMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
deserializationObjectMapper.findAndRegisterModules();
this.deserializationObjectMapper = deserializationObjectMapper;
}
public DataImportGatingConfig getGatingS3Config() {
try {
return getDataImportGatingConfigWithCache(gatingS3Config.getDataImportGatingObject());
} catch (Exception e) {
log.error(
"Error occurred to retrieve data-import-gating config file {} in bucket {}. Exception: {}. Error: {}",
gatingS3Config.getDataImportGatingObject(),
gatingS3Config.getDataImportGatingBucket(),
e.getClass().getName(),
e.getMessage());
return null;
}
}
/*
* Cache/retrieve objects related to data-import-gating, e.g. the email allowlist
*/
public String getDataImportGatingObjectAsString(String objectKey) {
try {
return getDataImportGatingObjectAsStringWithCache(objectKey);
} catch (Exception e) {
log.error(
"Error during data-import-gating object {} retrieval. Exception: {}. Error: {}",
objectKey,
e.getClass().getName(),
e.getMessage());
throw new DataImportException("Error retrieving data-import-gating object " + objectKey, e);
}
}
private String getDataImportGatingObjectAsStringWithCache(String objectKey) {
String dataImportGatingObject =
cacheService.get(RedisConfiguration.DATA_IMPORT_GATING_CACHE_NAME, objectKey, String.class);
if (dataImportGatingObject != null) {
return dataImportGatingObject;
}
dataImportGatingObject = getObjectAsString(objectKey);
cacheService.set(
RedisConfiguration.DATA_IMPORT_GATING_CACHE_NAME,
objectKey,
dataImportGatingObject,
gatingS3Config.getDataImportGatingExpiration());
return dataImportGatingObject;
}
private String getObjectAsString(String objectKey) {
return getObject(objectKey).asUtf8String();
}
private byte[] getObjectBytes(String objectKey) {
return getObject(objectKey).asByteArray();
}
private ResponseBytes<GetObjectResponse> getObject(String objectKey) {
String objectKeyWithEnv = ensureEnvironmentPrefixExists(objectKey);
GetObjectRequest getObjectRequest = GetObjectRequest.builder()
.bucket(gatingS3Config.getDataImportGatingBucket())
.key(objectKeyWithEnv)
.build();
ResponseBytes<GetObjectResponse> getObjectResponse = s3Client.getObjectAsBytes(getObjectRequest);
log.info("Successfully retrieved {} from S3", objectKeyWithEnv);
return getObjectResponse;
}
private DataImportGatingConfig getDataImportGatingConfigWithCache(String objKey) throws IOException {
DataImportGatingConfig config = cacheService.get(
RedisConfiguration.DATA_IMPORT_GATING_CACHE_NAME, objKey, DataImportGatingConfig.class);
if (config != null) {
return config;
}
byte[] configBytes = getObjectBytes(objKey);
if (configBytes == null) {
return null;
}
log.info("Data Import Gating Config: {}", new String(configBytes, Charset.forName("UTF-8")));
config = deserializationObjectMapper.readValue(configBytes, DataImportGatingConfig.class);
cacheService.set(
RedisConfiguration.DATA_IMPORT_GATING_CACHE_NAME,
objKey,
config,
gatingS3Config.getDataImportGatingExpiration());
return config;
}
private String ensureEnvironmentPrefixExists(String objectKey) {
return StringUtils.prependIfMissing(objectKey, gatingS3Config.getEnvironmentPrefix());
}
}

78
direct-file/backend/src/main/java/gov/irs/directfile/api/dataimport/gating/DataImportGatingEmailAllowlistService.java Normal file
View file

@ -0,0 +1,78 @@
package gov.irs.directfile.api.dataimport.gating;
import java.nio.charset.StandardCharsets;
import java.util.Arrays;
import java.util.Base64;
import java.util.Collections;
import java.util.Set;
import java.util.stream.Collectors;
import lombok.Getter;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.apache.pdfbox.util.Hex;
import org.bouncycastle.crypto.digests.SHA256Digest;
import org.bouncycastle.crypto.macs.HMac;
import org.bouncycastle.crypto.params.KeyParameter;
import org.springframework.stereotype.Service;
import gov.irs.directfile.api.config.DataImportGatingConfigurationProperties;
@Slf4j
@Service
public class DataImportGatingEmailAllowlistService {
private final DataImportGatingConfigService dataImportGatingConfigService;
@Getter
private final boolean allowlistEnabled;
private Set<String> allowlist;
private final String allowListObject;
private final byte[] hexKey;
public DataImportGatingEmailAllowlistService(
DataImportGatingConfigurationProperties configProps,
DataImportGatingConfigService dataImportGatingConfigService) {
this.dataImportGatingConfigService = dataImportGatingConfigService;
this.allowlistEnabled = configProps.getAllowlist().enabled();
this.allowListObject = configProps.getAllowlist().objectKey();
this.hexKey = Hex.decodeHex(configProps.getAllowlist().hexKey());
}
// determines whether the identity provider-supplied email address is on our allowlist
public boolean emailOnAllowlist(String email) {
if (allowlistEnabled) {
loadAllowlist(); // trigger cache reload if needed
String base64Mac = emailMac(email);
return allowlist.contains(base64Mac);
}
// allowlist disabled
log.info("Allowlist is disabled, so emailOnAllowlist is false");
return false;
}
private String emailMac(String email) {
HMac hMac = new HMac(new SHA256Digest());
hMac.init(new KeyParameter(hexKey));
byte[] in = StringUtils.lowerCase(email).getBytes(StandardCharsets.UTF_8);
hMac.update(in, 0, in.length);
byte[] hMacOut = new byte[hMac.getMacSize()];
hMac.doFinal(hMacOut, 0);
return Base64.getEncoder().encodeToString(hMacOut);
}
private void loadAllowlist() {
try {
this.allowlist = Arrays.stream(dataImportGatingConfigService
.getDataImportGatingObjectAsString(allowListObject)
.split("\n"))
.collect(Collectors.toSet());
log.info("Allowlist checked, total items: {}", allowlist.size());
} catch (Exception e) {
// should we set up an alert on this error?
log.error("Error during allowlist retrieval: {}", e.getMessage());
this.allowlist = Collections.emptySet();
}
}
}

108
direct-file/backend/src/main/java/gov/irs/directfile/api/dataimport/gating/DataImportGatingService.java Normal file
View file

@ -0,0 +1,108 @@
package gov.irs.directfile.api.dataimport.gating;
import java.security.SecureRandom;
import java.time.ZonedDateTime;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.stereotype.Service;
import gov.irs.directfile.api.config.DataImportGatingConfigurationProperties;
import gov.irs.directfile.api.config.DataImportGatingS3Properties;
@Slf4j
@Service
@EnableConfigurationProperties({DataImportGatingS3Properties.class, DataImportGatingConfigurationProperties.class})
public class DataImportGatingService {
@Value("${direct-file.data-import.enabled}")
private boolean dataImportEnabled;
private final DataImportGatingConfigService gatingConfigService;
private final DataImportGatingEmailAllowlistService emailAllowlistService;
private static final SecureRandom secureRandom = new SecureRandom();
public DataImportGatingService(
DataImportGatingConfigService gatingConfigService,
DataImportGatingEmailAllowlistService emailAllowlistService) {
this.gatingConfigService = gatingConfigService;
this.emailAllowlistService = emailAllowlistService;
}
public DataImportBehavior getBehavior(String currentUserEmail) {
if (!dataImportEnabled) {
return DataImportBehavior.DATA_IMPORT_ABOUT_YOU_BASIC;
}
// if allow-list disabled or not on allow-list, will continue with windowing
if (emailAllowlistService.emailOnAllowlist(currentUserEmail)) {
return DataImportBehavior.DATA_IMPORT_ABOUT_YOU_BASIC_PLUS_IP_PIN_PLUS_W2;
}
DataImportGatingConfig config = gatingConfigService.getGatingS3Config();
if (config == null) {
log.warn(
"Failed to retrieve data-import-gating config file, default to DataImportBehavior DATA_IMPORT_ABOUT_YOU_BASIC_PLUS_IP_PIN");
return DataImportBehavior.DATA_IMPORT_ABOUT_YOU_BASIC_PLUS_IP_PIN;
}
// if windowing is empty, assume all time available
if (config.getWindowing().isEmpty()) {
return pickPercentageBasedBehavior(config);
} else {
ZonedDateTime now = getCurrentTime();
boolean inWindow = config.getWindowing().stream()
.anyMatch(window -> now.isAfter(window.getStart()) && now.isBefore(window.getEnd()));
if (inWindow) {
return pickPercentageBasedBehavior(config);
} else {
log.warn(
"Data import not allowed outside the specified time windows: {}",
config.getWindowing().toString());
return DataImportBehavior.DATA_IMPORT_ABOUT_YOU_BASIC_PLUS_IP_PIN;
}
}
}
private DataImportBehavior pickPercentageBasedBehavior(DataImportGatingConfig config) {
// if percentage empty, assume default behavior (3)
if (config.getPercentages().isEmpty()) {
return DataImportBehavior.DATA_IMPORT_ABOUT_YOU_BASIC_PLUS_IP_PIN;
}
List<String> behaviorPool = new ArrayList<>();
int totalPercentage = 0;
// Populate the behavior pool based on percentages
for (DataImportGatingConfig.Percentage percentage : config.getPercentages()) {
int count = percentage.getPercentage();
if (count > 0) {
behaviorPool.addAll(Collections.nCopies(count, percentage.getBehavior()));
}
totalPercentage += count;
}
// Fill remaining percentage with default behavior
DataImportBehavior defaultBehavior = DataImportBehavior.DATA_IMPORT_ABOUT_YOU_BASIC_PLUS_IP_PIN;
if (totalPercentage < 100) {
behaviorPool.addAll(Collections.nCopies(100 - totalPercentage, defaultBehavior.name()));
}
// Shuffle and pick a random behavior
Collections.shuffle(behaviorPool);
String behaviorStr = behaviorPool.get(secureRandom.nextInt(behaviorPool.size()));
log.info("pickPercentageBasedBehavior: {}", behaviorStr);
return DataImportBehavior.valueOf(behaviorStr);
}
protected ZonedDateTime getCurrentTime() {
return ZonedDateTime.now();
}
}

137
direct-file/backend/src/main/java/gov/irs/directfile/api/dataimport/model/PopulatedData.java Normal file
View file

@ -0,0 +1,137 @@
package gov.irs.directfile.api.dataimport.model;
import java.util.Date;
import java.util.UUID;
import com.fasterxml.jackson.databind.JsonNode;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.EntityListeners;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import jakarta.persistence.Transient;
import lombok.Getter;
import lombok.Setter;
import org.hibernate.annotations.CreationTimestamp;
@Getter
@Entity
@Table(name = "populated_data")
@EntityListeners({PopulatedDataEntityListener.class})
public class PopulatedData {
@Id
@GeneratedValue(generator = "UUID4")
@Column(nullable = false, updatable = false)
private UUID id;
@Setter
@Column(name = "taxreturn_id", nullable = false)
private UUID taxReturnId;
@Setter
@Column(name = "source", nullable = false)
private String source;
@Setter
@Column(name = "tags", nullable = false)
private String tags;
@Setter
@Column(name = "data", nullable = false)
private String dataCipherText;
@Setter
@Transient
private JsonNode data;
@Setter
@Column(name = "raw_data", nullable = false)
private String rawDataCipherText;
@Setter
@Transient
private JsonNode rawResponseData;
@Setter
@Column(name = "created_at", nullable = false, updatable = false, columnDefinition = "DEFAULT CURRENT_TIMESTAMP")
@CreationTimestamp
private Date createdAt;
public static final String docsExampleObject =
"""
{
"timeSinceCreation": 10000,
"aboutYouBasic": {
"state": "success",
"createdAt": "2024-11-27T00:00:00Z",
"payload": {
"source": "SADI",
"tags": ["BIOGRAPHICAL"],
"createdDate": "2024-01-01",
"dateOfBirth": "1980-08-02",
"email": "Homer.Simpson@test.email",
"mobileNumber": "+12223334444",
"landlineNumber": null,
"firstName": "Lisa",
"middleInitial": "",
"lastName": "Simpson",
"streetAddress": "123 Sesame St",
"streetAddressLine2": null,
"city": "Springfield",
"stateOrProvence": "TN",
"postalCode": "37172"
}
},
"ipPin": {
"state": "success",
"createdAt": "2024-11-27T00:00:00Z",
"payload": {
"source": "IPPIN",
"tags": ["BIOGRAPHICAL"],
"hasIpPin": false
}
},
"w2s": {
"state": "success",
"createdAt": "2024-11-27T00:00:00Z",
"payload": [
{
"id": "1a657d0c-b676-4bbf-9d18-d9ecb8547d8d",
"source": "EDP",
"tags": ["W2"],
"ein": "001234567",
"firstName": "Lisa",
"employerAddress": {
"name": "Goods and Stuff",
"nameLine2": "",
"streetAddress": "7588 PEACH TREE ST",
"streetAddressLine2": "",
"city": "SPRINGFIELD",
"stateOrProvence": "TN",
"postalCode": "37172",
"country": "USA"
},
"controlNumber": "000011 R#/123",
"employeeAddress": null,
"wagesTipsOtherCompensation": "20000",
"federalIncomeTaxWithheld": "2000",
"socialSecurityWages": "20000",
"socialSecurityTaxWithheld": "1240",
"medicareWagesAndTips": "20000",
"medicareTaxWithheld": "290",
"socialSecurityTips": "",
"allocatedTips": "",
"dependentCareBenefits": "",
"box12s": [],
"statutoryEmployeeIndicator": false,
"thirdPartySickPayIndicator": false,
"retirementPlanIndicator": false,
"nonQualifiedPlans": ""
}
]
}
}
}
""";
}

40
direct-file/backend/src/main/java/gov/irs/directfile/api/dataimport/model/PopulatedDataEntityListener.java Normal file
View file

@ -0,0 +1,40 @@
package gov.irs.directfile.api.dataimport.model;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import jakarta.persistence.PostLoad;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import gov.irs.directfile.models.encryption.DataEncryptDecrypt;
import gov.irs.directfile.models.encryption.GenericStringEncryptor;
@Component
@Slf4j
public class PopulatedDataEntityListener {
private GenericStringEncryptor genericStringEncryptor;
private ObjectMapper objectMapper;
@Autowired
public void configure(DataEncryptDecrypt dataEncryptDecrypt, ObjectMapper objectMapper) {
genericStringEncryptor = new GenericStringEncryptor(dataEncryptDecrypt);
this.objectMapper = objectMapper;
}
@PostLoad
public <T extends PopulatedData> void decryptColumn(T populatedData) {
try {
String decrypted = genericStringEncryptor.convertToEntityAttribute(populatedData.getDataCipherText());
JsonNode jsonNode;
jsonNode = objectMapper.readTree(decrypted);
populatedData.setData(jsonNode);
} catch (Exception e) {
log.error(
"Failed to decrypt / parse data column in populated_data. Exception: {}. Error: {}",
e.getClass().getName(),
e.getMessage());
}
}
}

35
direct-file/backend/src/main/java/gov/irs/directfile/api/dataimport/model/RawResponseDecryptor.java Normal file
View file

@ -0,0 +1,35 @@
package gov.irs.directfile.api.dataimport.model;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Component;
import gov.irs.directfile.models.encryption.DataEncryptDecrypt;
import gov.irs.directfile.models.encryption.GenericStringEncryptor;
@Component
@Slf4j
public class RawResponseDecryptor {
private final GenericStringEncryptor genericStringEncryptor;
private final ObjectMapper objectMapper;
public RawResponseDecryptor(DataEncryptDecrypt dataEncryptDecrypt, ObjectMapper objectMapper) {
this.genericStringEncryptor = new GenericStringEncryptor(dataEncryptDecrypt);
this.objectMapper = objectMapper;
}
public JsonNode decryptRawResponse(PopulatedData populatedData) {
try {
String decrypted = genericStringEncryptor.convertToEntityAttribute(populatedData.getRawDataCipherText());
return objectMapper.readTree(decrypted);
} catch (Exception e) {
log.error(
"Failed to decrypt / parse data column in populated_data. Exception: {}. Error: {}",
e.getClass().getName(),
e.getMessage());
}
return null;
}
}

162
direct-file/backend/src/main/java/gov/irs/directfile/api/dataimport/model/WrappedPopulatedData.java Normal file
View file

@ -0,0 +1,162 @@
package gov.irs.directfile.api.dataimport.model;
import java.text.ParseException;
import java.util.*;
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.node.JsonNodeFactory;
import com.fasterxml.jackson.databind.node.ObjectNode;
import lombok.Getter;
import lombok.RequiredArgsConstructor;
@Getter
@RequiredArgsConstructor
@SuppressWarnings({"PMD.SimpleDateFormatNeedsLocale", "PMD.AvoidDuplicateLiterals", "PMD.LiteralsFirstInComparisons"})
public class WrappedPopulatedData {
private final Data data;
static final String EMPTY_JSON = "{}"; /* e.g. for RAW_DATA which cannot be empty or null */
@Getter
@RequiredArgsConstructor
public static class Data {
private final WrappedPopulatedDataNode aboutYouBasic;
private final WrappedPopulatedDataNode ipPin;
private final WrappedPopulatedDataNode w2s;
private final WrappedPopulatedDataNode f1099Ints;
private final WrappedPopulatedDataNode f1095a;
private final long timeSinceCreation;
}
@JsonCreator
public WrappedPopulatedData(JsonNode _data) throws ParseException {
// this is used to deserialize mock json files for the MockDataImportService
// there is some differences between how deserialization of the mock json files occurs for Docker and local
// the below check handles this
JsonNode data = _data;
if (!_data.get("data").isEmpty() || !_data.get("data").toString().isBlank()) {
data = _data.get("data");
}
JsonNode aboutYouBasicData = data.get("aboutYouBasic");
JsonNode ipPinData = data.get("ipPin");
JsonNode w2sData = data.get("w2s");
JsonNode f1099IntsData = data.get("f1099Ints");
JsonNode f1095aData = data.get(
"f1095a"); // the key in the mocks is f1095A to mimic the GET /populate response, but in the response
// from the Lambda it is f1095As
WrappedPopulatedDataNode aboutYouBasic = new WrappedPopulatedDataNode(
aboutYouBasicData.get("payload"),
aboutYouBasicData.get("createdAt").textValue(),
aboutYouBasicData.get("state").asText());
WrappedPopulatedDataNode ipPin = new WrappedPopulatedDataNode(
ipPinData.get("payload"),
ipPinData.get("createdAt").textValue(),
ipPinData.get("state").asText());
WrappedPopulatedDataNode w2s = new WrappedPopulatedDataNode(
w2sData.get("payload"),
w2sData.get("createdAt").textValue(),
w2sData.get("state").asText());
// f1099Ints is optional
WrappedPopulatedDataNode f1099Ints = null;
if (f1099IntsData != null) {
f1099Ints = new WrappedPopulatedDataNode(
f1099IntsData.get("payload"),
f1099IntsData.get("createdAt").textValue(),
f1099IntsData.get("state").asText());
} else {
f1099Ints = new WrappedPopulatedDataNode();
}
WrappedPopulatedDataNode f1095a = null;
if (f1095aData != null) {
f1095a = new WrappedPopulatedDataNode(
f1095aData.get("payload"),
f1095aData.get("createdAt").textValue(),
f1095aData.get("state").asText());
} else {
f1095a = new WrappedPopulatedDataNode();
}
long timeSinceCreation = data.get("timeSinceCreation").asInt();
this.data = new Data(aboutYouBasic, ipPin, w2s, f1099Ints, f1095a, timeSinceCreation);
}
public static WrappedPopulatedData from(List<PopulatedData> listData, Date taxReturnCreatedAt) {
WrappedPopulatedDataNode aboutYouBasic = new WrappedPopulatedDataNode();
WrappedPopulatedDataNode ipPin = new WrappedPopulatedDataNode();
WrappedPopulatedDataNode w2s = new WrappedPopulatedDataNode();
WrappedPopulatedDataNode f1099Ints = new WrappedPopulatedDataNode();
WrappedPopulatedDataNode f1095a = new WrappedPopulatedDataNode();
for (PopulatedData data : listData) {
switch (data.getSource()) {
case "SADI":
handleAboutYou(aboutYouBasic, data);
break;
case "IPPIN":
handleIPPin(ipPin, data);
break;
case "W2":
handleW2(w2s, data.getData(), data.getCreatedAt().toString());
break;
case "FORM_1099_INT":
handle1099Int(
f1099Ints,
data.getData().get("f1099Ints"),
data.getCreatedAt().toString());
break;
case "FORM_1095_A":
handle1095A(
f1095a,
data.getData().get("f1095As"),
data.getCreatedAt().toString());
break;
default:
}
}
return new WrappedPopulatedData(new Data(
aboutYouBasic, ipPin, w2s, f1099Ints, f1095a, calculateTimeSinceCreationInMs(taxReturnCreatedAt)));
}
private static void handleAboutYou(WrappedPopulatedDataNode aboutYouBasic, PopulatedData data) {
aboutYouBasic.setPayload(data.getData());
aboutYouBasic.setCreatedAt(data.getCreatedAt().toString());
aboutYouBasic.setState(WrappedPopulatedDataNodeState.SUCCESS.getState());
}
private static void handleIPPin(WrappedPopulatedDataNode ipPin, PopulatedData data) {
ipPin.setPayload(data.getData());
ipPin.setCreatedAt(data.getCreatedAt().toString());
ipPin.setState(WrappedPopulatedDataNodeState.SUCCESS.getState());
}
private static void handleW2(WrappedPopulatedDataNode w2s, JsonNode payload, String createdAt) {
w2s.setPayload(payload);
w2s.setCreatedAt(createdAt);
w2s.setState(WrappedPopulatedDataNodeState.SUCCESS.getState());
}
private static void handle1099Int(WrappedPopulatedDataNode f1099Ints, JsonNode payload, String createdAt) {
f1099Ints.setPayload(payload);
f1099Ints.setCreatedAt(createdAt);
f1099Ints.setState(WrappedPopulatedDataNodeState.SUCCESS.getState());
}
private static void handle1095A(WrappedPopulatedDataNode f1095a, JsonNode payload, String createdAt) {
JsonNode responseNode = new ObjectNode(new JsonNodeFactory(false));
boolean has1095A =
!payload.isNull() && !payload.isEmpty() && !payload.asText().equals(EMPTY_JSON);
((ObjectNode) responseNode).put("has1095A", has1095A);
f1095a.setPayload(responseNode);
f1095a.setCreatedAt(createdAt);
f1095a.setState(WrappedPopulatedDataNodeState.SUCCESS.getState());
}
private static long calculateTimeSinceCreationInMs(Date taxReturnCreatedAt) {
return Calendar.getInstance().getTime().getTime() - taxReturnCreatedAt.getTime();
}
}

22
direct-file/backend/src/main/java/gov/irs/directfile/api/dataimport/model/WrappedPopulatedDataNode.java Normal file
View file

@ -0,0 +1,22 @@
package gov.irs.directfile.api.dataimport.model;
import com.fasterxml.jackson.databind.JsonNode;
import lombok.Getter;
import lombok.RequiredArgsConstructor;
import lombok.Setter;
@Getter
@Setter
@RequiredArgsConstructor
public class WrappedPopulatedDataNode {
private JsonNode payload;
private String createdAt;
private String state = WrappedPopulatedDataNodeState.INCOMPLETE.getState();
public WrappedPopulatedDataNode(JsonNode payload, String createdAt, String state) {
this.payload = payload;
this.createdAt = createdAt;
this.state = state;
}
}

15
direct-file/backend/src/main/java/gov/irs/directfile/api/dataimport/model/WrappedPopulatedDataNodeState.java Normal file
View file

@ -0,0 +1,15 @@
package gov.irs.directfile.api.dataimport.model;
import lombok.Getter;
@Getter
public enum WrappedPopulatedDataNodeState {
SUCCESS("success"),
INCOMPLETE("incomplete");
private final String state;
WrappedPopulatedDataNodeState(String state) {
this.state = state;
}
}

12
direct-file/backend/src/main/java/gov/irs/directfile/api/dispatch/DispatchContext.java Normal file
View file

@ -0,0 +1,12 @@
package gov.irs.directfile.api.dispatch;
import lombok.AllArgsConstructor;
@AllArgsConstructor
public class DispatchContext {
public String pathToSubmissionXml;
public String pathToManifestXml;
public String pathToUserContext;
public String submissionId;
}

87
direct-file/backend/src/main/java/gov/irs/directfile/api/dispatch/DispatchQueueService.java Normal file
View file

@ -0,0 +1,87 @@
package gov.irs.directfile.api.dispatch;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Locale;
import java.util.Map;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.stereotype.Service;
import software.amazon.awssdk.services.sqs.SqsClient;
import software.amazon.awssdk.services.sqs.model.*;
import gov.irs.directfile.api.config.MessageQueueConfigurationProperties;
import gov.irs.directfile.models.Dispatch;
import gov.irs.directfile.models.message.MessageHeaderAttribute;
import gov.irs.directfile.models.message.QueueMessageHeaders;
import gov.irs.directfile.models.message.dispatch.DispatchMessageVersion;
import gov.irs.directfile.models.message.dispatch.VersionedDispatchMessage;
import gov.irs.directfile.models.message.dispatch.payload.AbstractDispatchPayload;
import gov.irs.directfile.models.message.dispatch.payload.DispatchPayloadV1;
@Service
@Slf4j
@EnableConfigurationProperties(MessageQueueConfigurationProperties.class)
public class DispatchQueueService {
private final String queueName;
private String queueUrl;
private final SqsClient sqsClient;
private final ObjectMapper mapper = new ObjectMapper();
public DispatchQueueService(
SqsClient sqsClient, MessageQueueConfigurationProperties messageQueueConfigurationProperties) {
this.sqsClient = sqsClient;
this.queueName = messageQueueConfigurationProperties.getDispatchQueue();
JavaTimeModule module = new JavaTimeModule();
DateFormat df = new SimpleDateFormat("yyyy-dd-MM HH:mm:ss", Locale.US);
mapper.setDateFormat(df);
mapper.registerModule(module);
}
public void enqueue(Dispatch dispatch) {
try {
AbstractDispatchPayload payload = new DispatchPayloadV1(dispatch);
VersionedDispatchMessage<AbstractDispatchPayload> queueMessage = new VersionedDispatchMessage<>(
payload,
new QueueMessageHeaders()
.addHeader(MessageHeaderAttribute.VERSION, DispatchMessageVersion.V1.getVersion()));
String dispatchJsonString = mapper.writeValueAsString(queueMessage);
if (queueUrl == null || StringUtils.isBlank(queueUrl)) {
GetQueueUrlResponse getQueueUrlResponse = sqsClient.getQueueUrl(
GetQueueUrlRequest.builder().queueName(queueName).build());
queueUrl = getQueueUrlResponse.queueUrl();
}
// tax return id must not be null, otherwise NPE in SendMessageRequest builder.
MessageAttributeValue attrTaxReturnId = MessageAttributeValue.builder()
.dataType("String")
.stringValue(dispatch.getTaxReturnId().toString())
.build();
SendMessageRequest sendMsgRequest = SendMessageRequest.builder()
.queueUrl(queueUrl)
.messageAttributes(Map.of("TAX-RETURN-ID", attrTaxReturnId))
.messageBody(dispatchJsonString)
.build();
SendMessageResponse sendMessageResponse = sqsClient.sendMessage(sendMsgRequest);
String messageId = sendMessageResponse != null ? sendMessageResponse.messageId() : "";
int statusCode = sendMessageResponse != null && sendMessageResponse.sdkHttpResponse() != null
? sendMessageResponse.sdkHttpResponse().statusCode()
: 0;
log.info("SQS Message sent with messageId: {} and httpStatusCode: {}", messageId, statusCode);
} catch (QueueDoesNotExistException | InvalidMessageContentsException e) {
log.error("Error sending message to SQS: {}", e.getMessage());
} catch (Exception e) {
log.error("Unknown exception in SQS dispatch from class: {}", e.getClass());
}
}
}

67
direct-file/backend/src/main/java/gov/irs/directfile/api/dispatch/DispatchService.java Normal file
View file

@ -0,0 +1,67 @@
package gov.irs.directfile.api.dispatch;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Locale;
import java.util.UUID;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule;
import jakarta.transaction.Transactional;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.stereotype.Service;
import gov.irs.directfile.api.config.MessageQueueConfigurationProperties;
import gov.irs.directfile.api.loaders.service.FactGraphService;
import gov.irs.directfile.api.taxreturn.models.TaxReturn;
import gov.irs.directfile.models.Dispatch;
@Service
@Transactional
@EnableConfigurationProperties(MessageQueueConfigurationProperties.class)
public class DispatchService {
private final FactGraphService factGraphService;
private final DispatchQueueService dispatchQueueService;
private final boolean isSqsMessageSendingEnabled;
private final ObjectMapper mapper = new ObjectMapper();
public DispatchService(
FactGraphService factGraphService,
DispatchQueueService dispatchQueueService,
MessageQueueConfigurationProperties messageQueueConfigurationProperties) {
this.factGraphService = factGraphService;
this.dispatchQueueService = dispatchQueueService;
this.isSqsMessageSendingEnabled = messageQueueConfigurationProperties.isSqsMessageSendingEnabled();
JavaTimeModule module = new JavaTimeModule();
DateFormat df = new SimpleDateFormat("yyyy-dd-MM HH:mm:ss", Locale.US);
this.mapper.setDateFormat(df);
this.mapper.registerModule(module); // Java 8 time not registered by default
}
public void enqueue(UUID userId, TaxReturn taxReturn, DispatchContext context) {
Dispatch dispatch = new Dispatch(
userId,
taxReturn.getId(),
context.pathToManifestXml,
context.pathToUserContext,
context.pathToSubmissionXml,
context.submissionId);
if (isSqsMessageSendingEnabled) {
dispatchQueueService.enqueue(dispatch);
}
}
public void internalOnlyEnqueue(UUID userId, TaxReturn taxReturn, DispatchContext context) {
Dispatch dispatch = new Dispatch(
userId,
taxReturn.getId(),
context.pathToManifestXml,
context.pathToUserContext,
context.pathToSubmissionXml,
context.submissionId);
if (isSqsMessageSendingEnabled) {
dispatchQueueService.enqueue(dispatch);
}
}
}

10
direct-file/backend/src/main/java/gov/irs/directfile/api/errors/ApiErrorResponse.java Normal file
View file

@ -0,0 +1,10 @@
package gov.irs.directfile.api.errors;
import java.util.Map;
import org.springframework.http.HttpStatusCode;
import gov.irs.directfile.api.taxreturn.ApiErrorKeys;
public record ApiErrorResponse(
HttpStatusCode status, String message, ApiErrorKeys apiErrorKey, Map<String, Object> body) {}

49
direct-file/backend/src/main/java/gov/irs/directfile/api/errors/ApiResponseStatusException.java Normal file
View file

@ -0,0 +1,49 @@
package gov.irs.directfile.api.errors;
import java.util.Map;
import lombok.Getter;
import lombok.Setter;
import org.springframework.http.HttpStatusCode;
import org.springframework.lang.Nullable;
import org.springframework.web.server.ResponseStatusException;
import gov.irs.directfile.api.taxreturn.*;
@Getter
@Setter
public class ApiResponseStatusException extends ResponseStatusException {
private ApiErrorKeys apiErrorKey;
public static final String docsExampleObject =
"""
{
"status": "",
"message": "",
"apiErrorKey": "",
"body": {}
}
""";
public ApiResponseStatusException(
HttpStatusCode status, @Nullable String reason, ApiErrorKeys apiErrorKey, @Nullable Throwable cause) {
this(status, reason, apiErrorKey, Map.of(), cause);
}
public ApiResponseStatusException(
HttpStatusCode status,
@Nullable String reason,
ApiErrorKeys apiErrorKey,
Map<String, Object> apiErrorDetail,
@Nullable Throwable cause) {
super(status, reason, cause);
this.apiErrorKey = apiErrorKey;
if (apiErrorDetail != null && !apiErrorDetail.isEmpty()) {
apiErrorDetail.entrySet().forEach(entry -> {
this.getBody().setProperty(entry.getKey(), entry.getValue());
});
}
}
}

11
direct-file/backend/src/main/java/gov/irs/directfile/api/errors/DefaultCaseException.java Normal file
View file

@ -0,0 +1,11 @@
package gov.irs.directfile.api.errors;
public class DefaultCaseException extends Exception {
public DefaultCaseException() {
super();
}
public DefaultCaseException(String message) {
super(message);
}
}

Some files were not shown because too many files have changed in this diff Show more