monorepo/LCU.API.Operations.IncentiveAPI

Running The API Locally

1. Pull down docker compose test environment setup

https://landmarkcu.visualstudio.com/Platform/_git/LCU.Setup.Internal.EnvironmentSetup run 'docker compose up at root of docker compose directory to start all required testing containers'

2. [OPTIONAL] Change Core Wrapper Target Instance As Needed Ex: If you are testing a change in core wrapper that will bring in more data for new feature update, point at local instance

• If you need up to date live data you can run core wrapper locally, and understand how to run the core wrapper against production locally in a read only manner.
• The Incentive API will only query for data and should never write anything to core, but you should always make sure nothing else is running on your local that may write to core through your local Core Wrapper instance.
• If you are unsure whether or not your local Core Wrapper instance is configured to run safely against production in a read only manager reach out to your manager before attempting to do so. is for Query only, do not use any post endpoints

3. Update WebApi Launch Settings As Needed

Update your launchsettings.json in the WebApi project similar to sample provided bellow as to not collide with your own port configurations for local app run instances

{
    "iisSettings": {
        "windowsAuthentication": false,
        "anonymousAuthentication": true,
        "iisExpress": {
            "applicationUrl": "http://localhost:22532",
            "sslPort": 44380
        }
    },
    "$schema": "http://json.schemastore.org/launchsettings.json",
    "profiles": {
        "IIS Express": {
            "commandName": "IISExpress",
            "launchBrowser": true,
            "launchUrl": "swagger",
            "environmentVariables": {
                "ASPNETCORE_ENVIRONMENT": "Local"
            }
        },
        "WepApi": {
            "commandName": "Project",
            "launchBrowser": true,
            "launchUrl": "swagger",
            "environmentVariables": {
                "ASPNETCORE_ENVIRONMENT": "Local"
            },
            "applicationUrl": "https://localhost:20001;http://localhost:20000"
        }
    }
}

4. Point IncentiveAPI to your local instance of CoreWrapper as needed. Ex: testing a change in core wrapper that will bring in more data for new feature update

        "Http": {
            "CoreWrapper": "http://localhost:5000", //<--<< Like this

To execute actions from the swagger page you'll need to get a bearer token through the browser and enter it in the auth field on the swagger page You will need to do this any time you are starting the application with a fresh database for the first time. You can do this through either your local instance of the UI or you can use QA/UAT/PROD since the Azure AD auth token will not be tied to an environment.

Auth token location in browser, copy value

Paste value into here on the API Swagger page, be sure to chop off the highlighted front part "Bearer "

You can now run an initial employee sync through swagger to get up and running

Application

This contains the service layer and repositories with business logic.

Domain

This contains the domain bounded entities that are saved to the DB, as well as the models and DTO objects used to link the service layer and the web api.

WebApi

This project implements the scaffolding and RESTful controllers by communicating with the services in the application layer via models.

Vocabulary

• TODO

Departments

This is how multiple departments works. A user can be assigned to multiple departments, all department assignments have a start and end date. They have a home department which cannot be changed except by changing their department in Day Force. Below is an explanation.

Assumptions:

• A users "Home Department" can technically change, meaning it also has a start/end date. This is correct.
• A user can be in multiple floating/guest departments, and they also have start/end dates. This is correct.

Problems:

• If an activity arrives without a department, find the user's current department and credit that department?
  • This is indeterminate - no way to determine department!
• If an activity arrives with a department, credit the user/department pair.
• If the user has more than one department covering the period, this points to crediting the department on the activity.

Example: If I'm covering someone at a branch and I cause activity there - three things that can happen.

Credit the activity to the Associate and their home department. Credit the activity to the Associate and their float department(s). Credit the activity to the Associate and Department that came with the activity.

Outcome: If an activity does not include a department, then we have a problem. We should require activity to enter the system with an OPERATOR and DEPARTMENT (EX: Someone did it, somewhere)

Calculating Payouts

To calculate a payout your start with a person (Employee ID). Then you follow a series of data collection steps to build up the inputs so that you can calculate payouts.

Gather

• Person - Employee ID
• Time Period - Usually a specific Month or Year
• Departments assigned to Employee Id for Time Period
• Commissions that apply to the person's role
• For The Time Period:
  • Personal Goals for that person, per department
  • Department Goals for that person, per department
  • Activity for that person
  • Activity for department, per department

Payout Profiles

The data involved in calculating a payout has no real parent or hierarchy to it. This means in order to create a database row for this data, many of the rows look exactly the same except for one or two columns. When you include the exclusion of creating a payout editor in the UI due to project time constraints; it didn't make sense to try and force non-tabular data into a table.

So the Payout Profiles were created; and they encompass the various way to pay a person for a given Product, Role, and range of progress on a goal. These could in the future be moved to a document store like Mongo, or moved into a tabular format give they won't be edited directly by a human - which means creating a UI based payout editor.

In the short to medium term, the Payout Profile provides a readable and testable compromise for tracking how a product, role, progress, amount combination pays out for a given situation.

Calculate

• For each Commission, calculate progress based on activities, this gives commission payout.
• For each Goal, calculate Target Vs Actual (progress) based on activities.
• For each Goal, select a payout calculator for the Goal Product and Person Role.
• For each Goal, calculate payout.

At the end of the process you should have person, departments, goals, commissions, activities, progress, payout rules, and payouts for a time period.

Day Force

Had to make some adjustments to the extract criteria condition orders after adding the 365 days for term date requirement. You may see records that are an active status but have a term date. This is a situation where the employee has been rehired, and as such the hire date will show as a date later than the term date. That is just how the fields are used in Dayforce.

Mapping Fields From Day Force

How do I determine business line for a department?

• If it has branch in the full name, set business line to branch
• Otherwise use Member Services

How do I determine branches?

• Split on "-"
• If you see branch then the department is what is on the left of the "-"
• If you see Headquarters then the department is what is listed on the right of the "-"

Syncing Activities

The ProcessorHistoryTable

The ProcessorHistory Database table holds a the last successful runtime of all importers as well as a 'Bookmark' of the last entry in an external system that was processed. There are two fields [LastProcessedRecordId] and [LastProcessedRecordTimestamp]. One or both of these can be used to query the latest entries in an external system. The fields that are used for each process are documented in the 'Go-live' section of this document.

These values have to be seeded on go live. If the values for all processes are not seeded, then activities related to tha process will not be imported.

Why don't you fall back to 0?

There could potentially be millions of records in DNA by the time this project goes live. The API serving this information sometimes times out after only hundreds of thousands of records.

Why don't you fall back to the start of the day/month/year?

Requiring manual entry, enables the system to start exactly where it needs to be at go-live.

Seeding the Database For DNA

Syncing activities from External Systems is done via a hosted service and the DnaActivitySyncService. The process determines where in DNAs activity database to begin its search using data from the ProcessorRunHistory table. On it's first run, this database will need to be seeded with a pre-determined activity number. Every process can be set to the same activity number as after the first run will settle into their respective place in DNAs activity table.

Go-Live Tasks

Seeding the Database

The ProcessorHistory table keeps a 'Bookmark' of which activities have already been processed in any external system. At go live the business will have to determine where in any given external system we should start searching from. Keep in mind that starting too far in the past could overwhelm any API that you are calling.

Process	MostSignificantBookmarkField	What to populate it with
ImportAtmCheckCardSalesFromDna	LastProcessedRecordTimestamp	An ActvDateTime for an Activity in the OsiBank.ACTV table.
ImportCheckingAccountSalesFromDna	LastProcessedRecordTimestamp	An ActvDateTime for an Activity in the OsiBank.ACTV table.
ImportDpEligibleLoanSalesFromDna	LastProcessedRecordTimestamp	An ActvDateTime for an Activity in the OsiBank.ACTV table.
ImportMortgageSalesFromDna	LastProcessedRecordTimestamp	An ActvDateTime for an Activity in the OsiBank.ACTV table.
ImportVehicleSalesFromDna	LastProcessedRecordTimestamp	An ActvDateTime for an Activity in the OsiBank.ACTV table.
ImportActivityManagerReferralsFromDna	LastProcessedRecordId	An ID for an Activity in the ActivityManager.LF_LEAD table.

**Note that the LastProcessedRecordTimestamp field is save as 'Unspecified' no conversions should be used on it. This is so it represents the exact timestamp as a string that is in its source system.

Feature Management

This application uses Microsoft's FeatureManagement package to enable/disable features in the code.

This is initialized in startup like so: services.AddFeatureManagement();

The feature flags are configured at their default location in any of the appsettings:

"FeatureFlags" : { "EnableControllers" : true, "RunHostedServices": false }After adding an item to the config it is suggested to add the Flag's string to the FeatureFlag.cs static class under the configuration folder of the main project.

Feature flags can be used in [FeatureGate(*nameOfFlag*)] attributes in controllers to enable and disable them dynamically by changing the config:

namespace LCU.API.Collections.RepoApi.Controllers
{
 [Route("api/[controller]")] [ApiController] [FeatureGate(FeatureFlags.EnableControllers)] public class RefundStatusController : RepoControllerBase {

After injecting IFeatureManagement they can also be used inline to prevent entire bloks of code from running:

 if (await featureManager.IsEnabledAsync(FeatureFlags.FeatureA)) { /* Run the following code */ }

Learn more about Microsoft.FeatureManagement : (https://docs.microsoft.com/en-us/azure/azure-app-configuration/use-feature-flags-dotnet-core?tabs=core5x)

Learn more about the history and use cases of Feature Management: (https://martinfowler.com/articles/feature-toggles.html)

Ef Migration

This project currently uses EF Migration to handle setting up the database instead of a DACPAC. The files efmigrate.ps1 and efupdate.ps1 contain sample instructions that you can use, and will continue to be updated. It's important to note which environment you want to run against, as well as making sure that inMemory = false.

Within the code, this migration is called with dbContext.Database.Migrate() in startup.cs

You can read about EF Migrations here: https://docs.microsoft.com/en-us/ef/core/managing-schemas/migrations/?tabs=dotnet-core-cli

Important Tags

The following are tags for historic changes in the codebase that may be of interest to future developers

Tag	Reason

Developer Notes

Goal Month Ranges

A Goal's Month Range in the IncentiveAPI run from a Start DateTime (midnight on the first of the month) to an End DateTime (One second before midnight the next month).

 goal.StartDate = DateTime.Now(); goal.EndDate = DateTime.Now().AddMonth(1).AddSeconds(-1); //Avoid AddMilliseconds(-1) as this can cause issues matching table entries down-stream.

Whenever comparing a Goal's Month Range to another Start DateTime or End DateTime to mitigate any issues that could arise from differing timestamps.

 if(goal.StartDate.Date == otherGoal.StartDate.Date){ /*...*/ }

TimeZones

Per ADR_0006 when a DateTime enters the system without a specified TimeZone it should be converted to UTC using the sources system's assumed timezone.

//Convert Uspecified DateTime to UTC by first defining the timezone as Central Time
TimeZoneInfo AssumedSourceTimezone = TimeZoneConverter.TZConvert.GetTimeZoneInfo("America/Chicago");
SourceCreated = TimeZoneInfo.ConvertTime(ActivityDateTime, AssumedSourceTimezone, TimeZoneInfo.Utc),

Note, there are 'Windows' Timezone Id strings and 'Linux' Timezone ID strings. The TimeZoneConverter nuget package allows us to use either interchangeably. Linux Type strings are preferred as it matches our hosting evnironment

Note, if the DateTime coming in is intended to be DATE only, we do not do any explicit conversion. Leave it uncasted.

Assumed Timezone Chart

Source System	Assumed Timezone	Activities (may be incomplete)
Dna OSIBANK Database	America/Chicago	New Checking New ATM Card New Check Card Debt Protection Life Debt Protection Diability Credit Card Sale
Dna Osibank Database - Loans	None Date Only	Mortgage Loan Vehicle Loan
Dna ActivityManager Database	America/Chicago	Direct Deposit Referral Vehicle Referral Mortgage Referral New Checking Referral Credit Card Balance Transfer Referral ATM Card Referral Check Card Referral
Allied GAP and MBP File	None Date Only	GAP Sale MBP sale
Indirect Vehicle Sale Report	To Be Determined	Indirect Vehicle Referral

DNA Activity Process

Here is the basic way an activity enters and saves in Activity Manager For DNA:

1. A Processor Runs and grabs new ExternalActivities for a particular product
2. For all External Activities we use a ExternalActivity Strategy to Map it to an ActivityManager Activity
   1. The most basic mapping can be found in each External Activity Model ExternalActivity.MapToActivity()
   2. The External Activity Strategy attempts to look up any values that our system holds the truth for
      1. Employee Information
      2. Department Information
   3. All Activity Arrives in the 'Created State'
3. This Incentive Manager Activity is Issued to its Product/Type topic to the Queue
4. An Activity Arrives from the Queue and We immediately try to save it
   1. The save will fail if we are missing required data and a Dead Activity is Created
5. If the save is successful we will then use the product/types AutoDecisionStrategy to try to update the state based on the current Activity Data
   1. If the decisioning fails the state will not be updated, and in the next morning we will retry decisioning on all created activities

Notes:

• Some activities that are saved to 'ManualReview' also have processes that attempt to update them to other states. These processors run every morning.
• File Processors work slightly differently (they have file rows not external activities). They enter the system differently and typically only consistently use step 4 and 5.
  • These activities still get saved to the dead activity queue if they are incomplete
  • These activities still immediately get AutoDecisisoned after saving

Audit Record Information:

Hi Richard,

ActivityTransfer (and Activity Status Changes in general) can be found in the following table: ActivityStateChanges The ActivityTransferAction table holds the current request status (if both parties have accepted the transfer, which dictates if the state should be changed).

If an Activity is pending transfer its state will be ‘PendingTransfer’ if the transfer is accepted its state will be ‘Transferred’. If the Transfer has been rejected the state will return to what it was before the ‘PendingTransfer’ request.

An example query to show an accepted transfer in QA is: SELECT * FROM [IncentiveManager].[dbo].[ActivityStateChanges] WHERE ActivityId='28603'

An example query to show a rejected transfer in QA is: SELECT * FROM [IncentiveManager].[dbo].[ActivityStateChanges] WHERE ActivityId= ‘35769’

Activity Approval – can be found in the ActivityStateChanges table as well and will have the status ‘Accepted’

These can come from automated acceptance criteria in the system such as the following example:

SELECT * FROM [IncentiveManager].[dbo].[ActivityStateChanges] WHERE ActivityId='484'.

They can also come from manual acceptance by employees which the following query is an example of: SELECT *FROM [IncentiveManager].[dbo].[ActivityStateChanges] WHERE ActivityId='636'

The ActivityStateChanges table has a ‘Notes’ field that captures various details on why the change occurred – ie. A sale occurred, an account was opened, etc.

Permissions Change – the permissions changes can be found in the UserPermissionHistory table which details if an extra permission was added to a specific user (separate from their current role permissions).

The PermissionAdded column dictates if a permission was added via the Boolean 0/1 and the PermissionRemoved column dictates if a permission was removed via the Boolean 0/1 – both cannot be set per a row. The Permission column holds the string literal for the permission that was added/removed, and the CreatedBy row holds who assigned the permission (currently the Source will always be ‘Manual Assignment’ and the CreatedByType will always be 0 as the system doesn’t assign permissions.

Example Query: SELECT * FROM [IncentiveManager].[dbo].[UserPermissionHistory] Where EmployeeId = ‘11748’

Department Change on A User – These can be found in the DeptAssignments table. If a user gets assigned to a new Department, it will be added to this table

Role Change on a user as well as Payout Percentage Change (SlidingPayout) (Role, SlidingPayout, EndDate) – if any of these currently supported changes occur to a user’s role assignments at a department it will be listed in the UserDepartmentHistory table. The table captures the UserDeparmentId a ForeignKey to the DepartmentAssignments Table for that unique assignment, the ‘FieldName’ (Role, SlidingPayout, EndDate) that changed, the ‘OldValue’ the ‘NewValue’ and the ‘CreatedBy’ – who made the change, and ‘Created’ when the change occurred.

Example Query: SELECT * FROM [IncentiveManager].[dbo].[UserDepartmentHistory] Where UserDepartmentId='1209'

DepartmentGoalChange and Personal Goal Change - can be found in the [GoalChanges] table. If a PersonId is present then the change was for a Person for the specific DepartmentId, if a PersonId is null then the change was for the entire Department

Revised Changes - 8/2024

To prepare for Azure Kubernetes migration the LCU.API.Operations.IncentiveAPI repository has been changed to include:

• Follow design patterns as a guide url for reference: https://landmarkcu.visualstudio.com/Platform/_git/LCU.Platform.PatternTemplates
• Removal of Projects that were used to help load data from XP2 before DNA launch. (DnaDayforceDataExtract, LegacyIncentiveImporter)
• Follow project layer structure and naming closely to design patterns
• Removal of EF migration for table defintion, replaced with dacpac
• Update versions to support net 6.0
• Added LCU nuget packages LcuHttpClient, LcuLogging, LcuStatusHealthCheck
• Remove some of the seeding logic, new fixed entries should be scripted
• Revise some configuration classes to populate with implementation
• Revise/add some microservices to call Data Access object classes

Selected Comments Removed from previous version - 8/2024

Authorization\RequirePermissionAttribute.cs Require permissions on a method

• Comma separated list of allowed permission codes

Configuration\FeatureFlags.cs

• These feature flags should match what is in the appsettings under "FeatureFlags" to prevent Magic Strings.

Startup

• https://docs.microsoft.com/en-us/azure/azure-app-configuration/use-feature-flags-dotnet-core?tabs=core5x
• This is not the right way to do feature flags, as this cannot be dynamically changed after startup. Until we add IFeatureManager to the queue lib, we will be unable to disable the HostedTaskScheduler With feature flags.

ActivityController.cs

• Detect user here with UserService. May need to pass user in from client in the future

DataIntegrityException.cs

• Exception to be used in tasks run to verify the integrity of the data in the database
• This was added following the application wide transition of AppUsers from using a composite key of EmployeeId + Email to a GUID
• Ideally this will become obsolete and marked for removal in the future once all data linking is strictly enforced to use the new Ids

DeadActivityExpection.cs

• Throw this exception when an activity is missing required data to enter the system normally.
• So far this is DepartmentId or EmployeeId. Use the message from this exception to save a 'note' in the Dead Activity entry.

NotAvailableDecisionStrategy.cs

• If there is no decision strategy available for a product/activity type combination (ex Check Card Referral), then this is thrown.
• You may have added a new type of product and forgotten to create a new strategy or register an existing strategy. See the services/autodecision folder.

ServerErrorException.cs

• Do not want to expose database and related exceptions to the controllers / web layer.
• The lower layers log what happened for developers, then return a general error; unless a more specific error can be returned in a way to allow the client to act (such as NotFound).

AlliedSaleRow.cs

• Represents a row in the CSV file provided by Allied for downloading. Rows represents GP and MBP sales.
• Contacts Used: [email protected], [email protected]
• Contacts Used: [email protected], [email protected]
• These constants are from the Allied CSV files, and the value should only be MBP or GAP

DateTimeService.cs

• Convert DNA TimeStamp that has unspecified timezone to UTC by assuming DNA Time is in America/Chicago TZ
• Some Timestamps from DNA may come with Timezone Data,

DefaultPayoutStrategy.cs

• The employee at this point may have multiple payouts for a product. For example they may have a personal payout for mortgage, and a department payout for mortgage. (MSC Team Lead)
• For each eligible payout, you may need to look at either department level goals or personal.

PayoutCalculateService.cs

• Gathers goals, activity, payout profiles, and commissions for an employee-department pair for a specified time period and calculates payouts providing an audited view.
• Calculates Commission line items for a set of gathered department payout data and user department
• Would like to restrict this to products they have activities for, but that is hard with gap using other product types/etc.

DashboardService.cs

• In order to not mix in the goal and activity repositories and keep the dependency tree clean, this function was moved to a new service
• To Do: DashboardService - Serve up report like structures for displaying information.

DnaActivitySyncService.cs

• TODO: This ought to be converted to UTC, No way to know what to/from though, as this is source dependent

EStatementService.cs

• We use LastSuccessfulRunTime instead of LastProcessedRecordId because delivery methods are not captured by activity number.

GoalService.cs

• There is potential for some unexpected behaviour in this method if you try to save a department goal and a personnel goal for a employee in the department at the same time.
• The 'resulting' goal for that person will depend on which goal change is processed last.
• The client should not allow this behaviour, so we will not do anything about it here. This may be a future enhancement.

UserService.cs

• To Do: check into Role Deprecation

Revised Changes - 1/2025

Mass Transit Changes

• Attempt to utilize LCU.ServiceFactory pattern to handle multiple DNAActivity processes
• Upgrade MassTransit from RealTime Queue logic
• Remove Hosted Services including TopicHostedServices, replace with services that can be scheduled in Dispatcher.

Configuration Changes

• Moved and Renamed select configurations to follow design patterns more closely.
• Some of the File operations for Action, Allied processes were removed to be handled in Dispatcher.

DateTimeExtensions.cs

• Comments Removed Start of Month
  • Midnight on the First of the Month' for any Given DateTime.
  • The first second of a given DateTime's month. DateTime for same year and month, but at day 1 time 00:00:00 End of Month
  • One second before Midnight the next month' for any Given DateTime.
  • The Final Second of a given DateTime's month. DateTime for same year and month, but on the last day (28,29,30,31) at time 12:59:59

NewAtmCheckCardActivity.cs

• Comments Removed
  • ATM and Check Cards are 'Other' Type. They should only cause referrals to pay out and not pay out on their own.

DateTimeService.cs

• Comments Removed Method getNextBonusPeriod
  • From Ben: The logic is that it will pay on the second paycheck in the month following the incentives being finalized.
  • Landmark pays every 2nd week on Friday.
  • The seed date must be a payday. Method PayPeriodInMonth
  • Given a pay period in a month, which one is it.

ActivityDecisionService.cs

• Comments Removed Method UpdateAndDecideDpEligibleLoanActivities
  • TODO: Add paid out filter to this method

ActivityService.cs

• Comments Removed Method SaveActivityAsync
  • Check for employee ID first as it can be required to find departmentId Method LogGapInformation
  • BE21 accounts typically contain Inc or Llc best measure to detect we don't save minor type Method GetActivitiesAsync
  • Get all activities from the repository. Can Take a while to run.
  • If you are sending this to the client, it is highly recommended that you use the Paginated Version of this method ListByFilterAsync(). Method GetDepartmentActivityForMonthRangeAsync
  • Determine when the month range provided overlaps with the month range the user was assigned to this department Method ValidateCanApplyActivityStateChange
  • New accepted date must be restricted to only months that have not paid out or one month from current date Method GetActivityAuthorizedCommands
  • Determine what a user can do to an activity, this would in an ideal system return the list of possible commands for the user :-/
  • But for now, it can still be awesome, it returns the state transitions the user can do.

IUserService.cs

• Comments Removed Method SyncDayForceEmployeeExisting
  • You can have more than one home dept, but not more than one "active / open ended".

DNAActivityServices.cs

• Topics/RealTime Queue Logic Removed All DNAActivityServices
  • DNA Activity Services should be running every 10 minutes, these can be separate Dispatcher jobs
  • Messaging can be added to total the amount of activities received to replace old "New Topic Received" comments
  • The previous version of DnaActivityService under a hosted ran a method to enqueue all 7 service types
  • If there were activities found it would run the TopicHostedService which logged topic received and then called the old method ofHandleIncomingTopic

DayforceClientService.cs

• Comments Removed Method GetHttpResponseAsync
  • Only if response 401, we are taking the request uri from the response of first request and completing the get.

TO DO: (at later date):

• We would want to remove the custom service exceptions in a future upgrade
• Refactor DBApplicationIntegrity service to return all results.
• Use LcuHttpClient with Dayforce Get data method. Remove TrustingHttpClient
• May want to look into an alternate endpoint or solution to avoid redirect with Dayforce report data.
• Remove SeedRoles logic, find alternative solution.
• Remove SystemAppUserConstants class and make that a configuration.
• Reorganize and remove service classes from the Persistence namespace. Example: ApplicationDbIntegrityService.cs
• Remove try/catch from the DAO classes to ensure the service layer is handling the try/catch appropriately.
• Remove extra logic in repository classes and move to appropriate service layer.
  • Example: ProcessorHistoryRepository.cs - if (newHistoryRecord.LastSuccessfulRunTime < existingProcess.LastSuccessfulRunTime)