MicroService World

Architecture
Management BusinessGit FlowSome cool paragraph Git FlowSome cool paragraph Application Technology
- Management
- Business
  Git FlowSome cool paragraph
- Application
- Technology
Technology
DatabasesElastic searchIndex Versioning Overview Indexes can be versioned. These versions will handle breaking data changes. To facilitate this the repositories will use an alias. This alias allows us to couple the most recent version of the version on an elastic level. The mechanism used for this is called reindex . Backup the current index To kick of our changes, we will first reindex the existing index. We will add a version number on the end.
- Databases
  Elastic searchIndex Versioning Overview Indexes can be versioned. These versions will handle breaking data changes. To facilitate this the repositories will use an alias. This alias allows us to couple the most recent version of the version on an elastic level. The mechanism used for this is called reindex . Backup the current index To kick of our changes, we will first reindex the existing index. We will add a version number on the end.
- Kubernetes
- Release management
  Release management is the process of versioning changes and moving new versions through all environments. Each environment has its characteristics and purpose. Additionally it’s important to have good release notes to track all the changes throughout the software lifecycle. Environments Production environment Our production environment is the most important environment. It houses paying customers and holds sensitive data. Security is a great concern here. The requirements of this environment are
- Open Source
- Tools
  GithubGithub provides a lot of functionality out of the box for a small fee. It is true that choosing other providers might offer additional functionality, but this mostly comes at a price. As an enterprise the decision really depends on the added value of the functionality. This mostly boils down to reporting, visualization and ease of use. In beginning projects I would advocate to use Github and the functionality it offers to the fullest.
Projects
TradingTrading BotFor this project we will be using Freqtrade. An open source trading bot. With some very nice features, including backtesting, parameter optimization and strategy definitions. We can run it locally in a docker container. FreqtradeDocumentation Running the software To run the file we need a Dockerfile and a docker-compose.yml file. Dockerfile 1 2 3 4 FROM freqtradeorg/freqtrade:stable AS base WORKDIR /freqtrade EXPOSE 8080 COPY --chown=ftuser:ftuser . . Docker compose file 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 --- services: freqtrade: image: freqtradeorg/freqtrade:stable restart: unless-stopped container_name: freqtrade volumes: - ".
- Trading
  Trading BotFor this project we will be using Freqtrade. An open source trading bot. With some very nice features, including backtesting, parameter optimization and strategy definitions. We can run it locally in a docker container. FreqtradeDocumentation Running the software To run the file we need a Dockerfile and a docker-compose.yml file. Dockerfile 1 2 3 4 FROM freqtradeorg/freqtrade:stable AS base WORKDIR /freqtrade EXPOSE 8080 COPY --chown=ftuser:ftuser . . Docker compose file 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 --- services: freqtrade: image: freqtradeorg/freqtrade:stable restart: unless-stopped container_name: freqtrade volumes: - ".
- HRM
  ComponentsComponents form the basis of our HRM system. Each component represents compensation and benefits an employee can receive as part of their salary package. This document describes what components are and how they are used throughout the entire system. As you will see components relate closely to policies, the only difference being that components are based on law, sectoral and institutional rules. They are defined by law and therefor informatively enforce those laws.
Resources

Architecture

Management
Business
Git FlowSome cool paragraph
- Git Flow
  Some cool paragraph
Application
Technology

Management

Business

Git Flow
Some cool paragraph

Git Flow

Some cool paragraph

Application

Technology

Databases
Elastic searchIndex Versioning Overview Indexes can be versioned. These versions will handle breaking data changes. To facilitate this the repositories will use an alias. This alias allows us to couple the most recent version of the version on an elastic level. The mechanism used for this is called reindex . Backup the current index To kick of our changes, we will first reindex the existing index. We will add a version number on the end.
- Elastic search
  Index Versioning Overview Indexes can be versioned. These versions will handle breaking data changes. To facilitate this the repositories will use an alias. This alias allows us to couple the most recent version of the version on an elastic level. The mechanism used for this is called reindex . Backup the current index To kick of our changes, we will first reindex the existing index. We will add a version number on the end.
Kubernetes
Release management
Release management is the process of versioning changes and moving new versions through all environments. Each environment has its characteristics and purpose. Additionally it’s important to have good release notes to track all the changes throughout the software lifecycle. Environments Production environment Our production environment is the most important environment. It houses paying customers and holds sensitive data. Security is a great concern here. The requirements of this environment are
Open Source
Tools
GithubGithub provides a lot of functionality out of the box for a small fee. It is true that choosing other providers might offer additional functionality, but this mostly comes at a price. As an enterprise the decision really depends on the added value of the functionality. This mostly boils down to reporting, visualization and ease of use. In beginning projects I would advocate to use Github and the functionality it offers to the fullest.
- Github
  Github provides a lot of functionality out of the box for a small fee. It is true that choosing other providers might offer additional functionality, but this mostly comes at a price. As an enterprise the decision really depends on the added value of the functionality. This mostly boils down to reporting, visualization and ease of use. In beginning projects I would advocate to use Github and the functionality it offers to the fullest.
- Gita
  Gita is a wonderful tool that helps in managing multiple repositories as one. If you find yourself doing a lot of git actions through the command line. Check out this awesome tool. GithubGita Gita: a command-line tool to manage multiple git repos This tool has two main features display the status of multiple git repos such as branch, modification, commit message side by side (batch) delegate git commands/aliases and shell commands on repos from any working directory In this screenshot, the gita ll command displays the status of all repos.

1

printf("Hello World!");

1

printf("Hello World!");

$$\left( \sum_{k=1}^n a_k b_k \right)^2 \leq \left( \sum_{k=1}^n a_k^2 \right) \left( \sum_{k=1}^n b_k^2 \right)$$

---
title: Example Diagram
---
graph LR;
A[Hard edge] -->|Link text| B(Round edge)
B --> C{<strong>Decision</strong>}
C -->|One| D[Result one]
C -->|Two| E[Result two]

%%{init:{"fontFamily":"monospace", "sequence":{"showSequenceNumbers":true}}}%%
sequenceDiagram
Alice->>John: Hello John, how are you?
loop Healthcheck
John->>John: Fight against hypochondria
end
Note right of John: Rational thoughts!
John-->>Alice: Great!
John->>Bob: How about you?
Bob-->>John: Jolly good!

classDiagram
    Animal <|-- Duck
    Animal <|-- Fish
    Animal <|-- Zebra
    Animal : +int age
    Animal : +String gender
    Animal: +isMammal()
    Animal: +mate()
    class Duck{
      +String beakColor
      +swim()
      +quack()
    }
    class Fish{
      -int sizeInFeet
      -canEat()
    }
    class Zebra{
      +bool is_wild
      +run()
    }

%%{init:{"theme":"forest"}}%%
erDiagram
CUSTOMER }|..|{ DELIVERY-ADDRESS : has
CUSTOMER ||--o{ ORDER : places
CUSTOMER ||--o{ INVOICE : "liable for"
DELIVERY-ADDRESS ||--o{ ORDER : receives
INVOICE ||--|{ ORDER : covers
ORDER ||--|{ ORDER-ITEM : includes
PRODUCT-CATEGORY ||--|{ PRODUCT : contains
PRODUCT ||--o{ ORDER-ITEM : "ordered in"

journey
title My working day
section Go to work
Make tea: 5: Me
Go upstairs: 3: Me
Do work: 1: Me, Cat
section Go home
Go downstairs: 5: Me
Sit down: 3: Me

gantt
dateFormat  YYYY-MM-DD
title Adding GANTT diagram functionality to Mermaid
section A section
Completed task            :done,    des1, 2014-01-06,2014-01-08
Active task               :active,  des2, 2014-01-09, 3d
Future task               :         des3, after des2, 5d
Future task2              :         des4, after des3, 5d
section Critical tasks
Completed task in the critical line :crit, done, 2014-01-06,24h
Implement parser and jison          :crit, done, after des1, 2d
Create tests for parser             :crit, active, 3d
Future task in critical line        :crit, 5d
Create tests for renderer           :2d
Add to Mermaid                      :1d

pie title Pets adopted by volunteers
"Dogs" : 386
"Cats" : 85
"Rats" : 15

quadrantChart
    title Reach and engagement of campaigns
    x-axis Low Reach --> High Reach
    y-axis Low Engagement --> High Engagement
    quadrant-1 We should expand
    quadrant-2 Need to promote
    quadrant-3 Re-evaluate
    quadrant-4 May be improved
    Campaign A: [0.3, 0.6]
    Campaign B: [0.45, 0.23]
    Campaign C: [0.57, 0.69]
    Campaign D: [0.78, 0.34]
    Campaign E: [0.40, 0.34]
    Campaign F: [0.35, 0.78]

requirementDiagram

    requirement test_req {
    id: 1
    text: the test text.
    risk: high
    verifymethod: test
    }

    element test_entity {
    type: simulation
    }

    test_entity - satisfies -> test_req

gitGraph
commit
commit
branch develop
checkout develop
commit
commit
checkout main
merge develop
commit
commit

C4Context
title System Context diagram for Internet Banking System
Enterprise_Boundary(b0, "BankBoundary0") {
Person(customerA, "Banking Customer A", "A customer of the bank, with personal bank accounts.")
Person(customerB, "Banking Customer B")
Person_Ext(customerC, "Banking Customer C", "desc")

    Person(customerD, "Banking Customer D", "A customer of the bank, <br/> with personal bank accounts.")

    System(SystemAA, "Internet Banking System", "Allows customers to view information about their bank accounts, and make payments.")

    Enterprise_Boundary(b1, "BankBoundary") {

        SystemDb_Ext(SystemE, "Mainframe Banking System", "Stores all of the core banking information about customers, accounts, transactions, etc.")

        System_Boundary(b2, "BankBoundary2") {
        System(SystemA, "Banking System A")
        System(SystemB, "Banking System B", "A system of the bank, with personal bank accounts. next line.")
        }

        System_Ext(SystemC, "E-mail system", "The internal Microsoft Exchange e-mail system.")
        SystemDb(SystemD, "Banking System D Database", "A system of the bank, with personal bank accounts.")

        Boundary(b3, "BankBoundary3", "boundary") {
        SystemQueue(SystemF, "Banking System F Queue", "A system of the bank.")
        SystemQueue_Ext(SystemG, "Banking System G Queue", "A system of the bank, with personal bank accounts.")
        }
    }
    }

    BiRel(customerA, SystemAA, "Uses")
    BiRel(SystemAA, SystemE, "Uses")
    Rel(SystemAA, SystemC, "Sends e-mails", "SMTP")
    Rel(SystemC, customerA, "Sends e-mails to")

    UpdateElementStyle(customerA, $fontColor="red", $bgColor="grey", $borderColor="red")
    UpdateRelStyle(customerA, SystemAA, $textColor="blue", $lineColor="blue", $offsetX="5")
    UpdateRelStyle(SystemAA, SystemE, $textColor="blue", $lineColor="blue", $offsetY="-10")
    UpdateRelStyle(SystemAA, SystemC, $textColor="blue", $lineColor="blue", $offsetY="-40", $offsetX="-50")
    UpdateRelStyle(SystemC, customerA, $textColor="red", $lineColor="red", $offsetX="-50", $offsetY="20")

    UpdateLayoutConfig($c4ShapeInRow="3", $c4BoundaryInRow="1")

mindmap
  root((mindmap))
    Origins
      Long history
      ::icon(fa fa-book)
      Popularisation
        British popular psychology author Tony Buzan
    Research
      On effectiveness<br/>and features
      On Automatic creation
        Uses
            Creative techniques
            Strategic planning
            Argument mapping
    Tools
      Pen and paper
      Mermaid

timeline
title History of Social Media Platform
2002 : LinkedIn
2004 : Facebook
: Google
2005 : Youtube
2006 : Twitter

sankey-beta

%% source,target,value
Electricity grid,Over generation / exports,104.453
Electricity grid,Heating and cooling - homes,113.726
Electricity grid,H2 conversion,27.14

xychart-beta
title "Sales Revenue"
x-axis [jan, feb, mar, apr, may, jun, jul, aug, sep, oct, nov, dec]
y-axis "Revenue (in $)" 4000 --> 11000
bar [5000, 6000, 7500, 8200, 9500, 10500, 11000, 10200, 9200, 8500, 7000, 6000]
line [5000, 6000, 7500, 8200, 9500, 10500, 11000, 10200, 9200, 8500, 7000, 6000]

block-beta
columns 1
db(("DB"))
blockArrowId6<["&nbsp;&nbsp;&nbsp;"]>(down)
block:ID
A
B["A wide one in the middle"]
C
end
space
D
ID --> D
C --> D
style B fill:#969,stroke:#333,stroke-width:4px


---
title: "TCP Packet"
---
packet-beta
0-15: "Source Port"
16-31: "Destination Port"
32-63: "Sequence Number"
64-95: "Acknowledgment Number"
96-99: "Data Offset"
100-105: "Reserved"
106: "URG"
107: "ACK"
108: "PSH"
109: "RST"
110: "SYN"
111: "FIN"
112-127: "Window"
128-143: "Checksum"
144-159: "Urgent Pointer"
160-191: "(Options and Padding)"
192-255: "Data (variable length)"

architecture-beta
group api(cloud)[API]

    service db(database)[Database] in api
    service disk1(disk)[Storage] in api
    service disk2(disk)[Storage] in api
    service server(server)[Server] in api

    db:L -- R:server
    disk1:T -- B:server
    disk2:T -- B:db

Databases

Elastic search
Index Versioning Overview Indexes can be versioned. These versions will handle breaking data changes. To facilitate this the repositories will use an alias. This alias allows us to couple the most recent version of the version on an elastic level. The mechanism used for this is called reindex . Backup the current index To kick of our changes, we will first reindex the existing index. We will add a version number on the end.

Elastic search

Index Versioning

Overview

Indexes can be versioned. These versions will handle breaking data changes. To facilitate this the repositories will use an alias. This alias allows us to couple the most recent version of the version on an elastic level.

The mechanism used for this is called reindex .

Backup the current index

To kick of our changes, we will first reindex the existing index. We will add a version number on the end. In this example we will call this version 1.

To reindex use the following request:

POST _reindex
{
  "source": {
    "index": "my-index-000001"
  },
  "dest": {
    "index": "my-new-index-000001"
  }
}

Additionally we want to remove fields we don’t want in our new index. This will allow us to create a clean index when needed. To exclude fields we can use the following syntax. By using the script we can remove fields we don’t want in the new index

POST /_reindex
{
  "source": {
    "index": "my-index-000001"
  },
  "dest": {
    "index": "my-index-000001"
  },
  "conflicts": "proceed",
  "script" : {
    "source": "ctx._source.data.remove('properties.review_meta')"
  }

}

Create an alias

We will create an alias. This alias can be used in the application. We can change this alias when we need changed the index underneath. This will prevent the need to update the application on a data change.

POST _aliases
{
  "actions": [
    {
      "add": {
        "index": "my-data-stream",
        "alias": "my-alias"
      }
    }
  ]
}

Kubernetes

Release management

Release management is the process of versioning changes and moving new versions through all environments. Each environment has its characteristics and purpose. Additionally it’s important to have good release notes to track all the changes throughout the software lifecycle.

Environments

Production environment

Our production environment is the most important environment. It houses paying customers and holds sensitive data. Security is a great concern here.

The requirements of this environment are

Performance: the cluster that supports this environment will be larger and faster to assure smooth and fast operation for our customer
Stable software versions: Only stable software will be released here. No alpha, beta or other unofficial software is allowed.
Client only: No test, demo, development client frontends are allowed here
Backup: Intensive backup strategies will be used to safe guard client data
Scalability: The cluster will allow for autoscaling to support even the most intensive workloads
Security: Security in this cluster will be tested and improved to provide trust towards our clients
Monitoring & alerting: this environment will have extensive monitoring and alerting so smooth operation can be assured
Availability: this environment strives towards 99,889% availability
Rolling release: Release will be done without downtime

Test/staging/acceptance environment

A test environment is used for end-to-end testing. New releases are prepared through the test environment. This environment will contain meaningful data. Testers will test the application as it would be used by customers.

This environment will contain the latest versions that are considered to be released to production. Therefor it is also ideal for demo environments.

Data management in this environment will be handled differently. To provide a controlled environment the dataset used will be defined as a standard. This dataset will be reset periodically to assure consistency while testing, and provide a standard state for demos.

Characteristics of the environment are:

Data:
- GDPR compliant data will be used
- Meaning full data is required
- Data will be reset periodically
- Data will be provided by business
Beta software: beta software will be used when available for a given component. In other cases the latest stable release will be used.
Backup: on a data level datasets will be used, those datasets will be backed up and restored on a given interval
Clients:
- No production data is allowed
- Customer test environments can be hosted here
- Demo environments must be hosted here
- Test environments must be hosted here
Testing:
- Automated integration testing will be done continuously
- Automated smoke testing will be done continuously
- Testers will manually test and confirm release requirements
- Testers will expand the automated tests to account for new functionality
Scalability / availability: This environment will have limitations on scaling to prevent unnecessary costs
Rolling releases: changes to this environment will be created automatically when new beta packages are created

Development environment

When developers create new features and push changes to the develop branch, new alpha releases are automatically created. These releases will make sure the development environment is always in the latest state containing all changes. This allows developers to test their features in the context of other components.

Data: data here will not represent real data. Developers use this to test functionality hundreds of times. To maintain and manage realistic data would only slow down the development process. Its the responsibility of the testers to do end-to-end testing.
Alpha software: this environment will run only alpha software.
Backup: this environment will not be backed up
Clients:
- Developers will setup clients as is needed, conform their development requirements
Scalability / availability: This environment will have strict limitations on scaling to prevent unnecessary costs
Rolling releases: changes to this environment will be created automatically when new alpha packages are created

Standards

Semantic Versioning

We are using semantic versioning. This technique is used in almost all modern software from your browser, to version of the operating system on your phone.

More information about semantic versioning can be found here:

https://semver.org/lang/nl/

Git-flow

Git flow is a strategy on how to handle code changes and updates to software. To achieve the semantic versioning, git flow is used to handle code in the source control system.

More information can be found here: https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow

Git-ops

Git ops is a technique, where code changes result in deployable containers. Secondly the desired state of an environment is defined in code. When there is a delta between the desired state and the actual state in the environment, git ops will reconcile these changes until the desired state is achieved.

Conventional commits

Tracking the changes we make in code and creating meaningful release notes can be very tedious. We will be using a technique called conventional commits to create release notes automatically.

To achieve good release notes, programmers and release managers will add simple keywords to each commit message. These keywords will be parsed on releasing the software and will be converted into meaningful release notes automatically.

Information about conventional commits can be found here: https://www.conventionalcommits.org/en/v1.0.0/

Implications

Project management

Every part of the application is managed and released without context to another. Meaning they can be developed separately. This has these advantages:

Projects become smaller and more manageable
Project teams become smaller and allow for better planning
Smaller teams allows for easier alignment
Output of an API project becomes the input/analysis for a frontend project
Projects can be outsourced more easily
Smaller projects have more reliable plannings
Backends allow for integrations even when frontend is not ready

Open Source

Tools

Github
Github provides a lot of functionality out of the box for a small fee. It is true that choosing other providers might offer additional functionality, but this mostly comes at a price. As an enterprise the decision really depends on the added value of the functionality. This mostly boils down to reporting, visualization and ease of use. In beginning projects I would advocate to use Github and the functionality it offers to the fullest.
Gita
Gita is a wonderful tool that helps in managing multiple repositories as one. If you find yourself doing a lot of git actions through the command line. Check out this awesome tool. GithubGita Gita: a command-line tool to manage multiple git repos This tool has two main features display the status of multiple git repos such as branch, modification, commit message side by side (batch) delegate git commands/aliases and shell commands on repos from any working directory In this screenshot, the gita ll command displays the status of all repos.

Github

Github provides a lot of functionality out of the box for a small fee. It is true that choosing other providers might offer additional functionality, but this mostly comes at a price. As an enterprise the decision really depends on the added value of the functionality. This mostly boils down to reporting, visualization and ease of use. In beginning projects I would advocate to use Github and the functionality it offers to the fullest.

Github Actions

Github actions allow us to automate a lot of things, that would otherwise require more expensive tools. The trade of is, that it is not a point and click interface. The advantage is, that it is widely used and therefor a lot of documentation can be found online. Github actions allow us to build and deploy our application automatically after changes are pushed to a branch.

NuGet Registry

GithubNuGet registry

To automatically create a NuGet package from code, all we need to do is create a file .github/forkflows/release.yaml from the root of our project.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99


name: Build Nuget Packages
on:
  workflow_dispatch: # Allow running the workflow manually from the GitHub UI
  push:
    branches:
      - 'main'

env:
  NuGetDirectory: ${{ github.workspace}}/nuget

jobs:
  create_nuget:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0 # Get all history to allow automatic versioning using MinVer
      
      # Install the .NET SDK indicated in the global.json file
      - name: Setup .NET
        uses: actions/setup-dotnet@v3
        with:
          dotnet-version: 7.0.x #${{ matrix.dotnet-version }}

      - name: Update path
        run: export PATH="/usr/share/dotnet:$PATH"

      - name: Install dependencies
        run: dotnet restore

      - name: Build
        run: dotnet build --configuration Release --no-restore
      
      #      - name: Test
      #        run: dotnet test --no-restore --verbosity normal
      
      
      # Create the NuGet package in the folder from the environment variable NuGetDirectory
      - run: dotnet pack --configuration Release --output ${{ env.NuGetDirectory }}
      
      # Publish the NuGet package as an artifact, so they can be used in the following jobs
      - uses: actions/upload-artifact@v3
        with:
          name: nuget
          if-no-files-found: error
          retention-days: 7
          path: ${{ env.NuGetDirectory }}/*.nupkg

      - name: Display generated packages
        run: ls -la ${{ env.NuGetDirectory }}/*.nupkg
  

  run_test:
    runs-on: ubuntu-latest
    needs: [ create_nuget ]
    steps:
      - uses: actions/checkout@v3
      - name: Setup .NET
        uses: actions/setup-dotnet@v3
        with:
          dotnet-version: 7.0.x #${{ matrix.dotnet-version }}

      - name: Update path
        run: export PATH="/usr/share/dotnet:$PATH"
      - name: Run tests
        run: dotnet test --configuration Release

  deploy:
    # Publish only when creating a GitHub Release
    # https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository
    # You can update this logic if you want to manage releases differently
    #if: github.event_name == 'release'
    runs-on: ubuntu-latest
    needs: [ run_test ]
    steps:
      # Download the NuGet package created in the previous job
      - uses: actions/download-artifact@v3
        with:
          name: nuget
          path: ${{ env.NuGetDirectory }}

      # Install the .NET SDK indicated in the global.json file
      - name: Setup .NET Core
        uses: actions/setup-dotnet@v3
        with:
          dotnet-version: 7.0.x #${{ matrix.dotnet-version }}

      - name: Update path
        run: export PATH="/usr/share/dotnet:$PATH"

      # Publish all NuGet packages to NuGet.org
      # Use --skip-duplicate to prevent errors if a package with the same version already exists.
      # If you retry a failed workflow, already published packages will be skipped without error.
      - name: Publish NuGet package
        shell: pwsh
        run: |
          foreach($file in (Get-ChildItem "${{ env.NuGetDirectory }}" -Recurse -Include *.nupkg)) {
              dotnet nuget push $file --api-key "${{ secrets.NUGET_STORE_API_KEY }}" --source "${{ secrets.NUGET_STORE }}" --skip-duplicate
          }          

Important

Versioning is not done automatically. You need to manually set your version in the project file in your solution. The build will not fail, it will not push the new version when it already exists on the NuGet registry.

Optional: Remove the parameter --skip-duplicate from the Publish NuGet package command.

Explaining the code:

line 5: Script will run when changes are pushed to the main branch
line 9: Sets the nuget directory for the build, this will not influence your build.
line 23: This example depends on dotnet 7.0.x, where x is a wildcard to get the most recent version.
line 32: Runs the build with the release option enabled.
line 99: The option --skip-duplicate is used to prevent failure when the version already exists. Remove this if you want the build to fail on duplicate versions.

Note

Before uploading this, make sure to create the secrets in your github account.

NUGET_STORE_API_KEY: Your api key with access to the NuGet registry
NUGET_STORE: Url to your NuGet registry

Docker container registry

GithubDocker registry

We can create docker containers and deploy them to the Github container registry easily.

Note

You can easily deploy your containers from Github actions. Personally I prefer Gitops where we just create the container. Another service will pick it up and will deploy it.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136


name: Release Workflow

on:
  push:
    branches:
      - 'release/*.*.*'
      - 'main'
      - 'develop'

env:
  ASPNETCORE_ENVIRONMENT: Development
  REGISTRY: "ghcr.io/microservice-world"
  IMAGE_NAME: "my-api"
  REGISTRY_USER: my-github-username
jobs:
  version:
    name: Determine version
    runs-on: ubuntu-latest
    outputs:
      branchName:            ${{ steps.version_step.outputs.branchName }}
      fullSemVer:            ${{ steps.version_step.outputs.fullSemVer }}
  
      GitVersion_BranchName: ${{ steps.version_step.outputs.GitVersion_BranchName }}
      GitVersion_SemVer: ${{ steps.version_step.outputs.GitVersion_SemVer }}
    steps:
      - name: Checkout
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Install GitVersion
        uses: gittools/actions/gitversion/setup@v1.1.1
        with:
          versionSpec: '5.x'

      - name: Determine Version
        id: version_step # step id used as reference for output values
        uses: gittools/actions/gitversion/execute@v1.1.1
          
      - name: Display GitVersion variables (without prefix)
        run: |
             echo "SemVer (env.semVer)            : ${{ steps.version_step.outputs.semVer }}"             
    
    
  build:
    name: build
    needs: version
    runs-on: ubuntu-latest
    outputs:
      DockerImage: "${{ env.IMAGE_NAME}}:${{ needs.version.outputs.GitVersion_SemVer }}"
      DockerImageUrl: "https://github.com/${{ github.repository }}/pkgs/container/${{ env.IMAGE_NAME}}"
    steps:
      - name: Checkout the repo
        uses: actions/checkout@v3

      - name: Build container image
        run:  |
          docker build  \
              --no-cache \
              -f Dockerfile \
              -t $(echo $REGISTRY)/$(echo $IMAGE_NAME):${{ needs.version.outputs.GitVersion_SemVer }} \
              -t $(echo $REGISTRY)/$(echo $IMAGE_NAME):latest \
              --label "org.opencontainers.image.source=https://github.com/$(echo $REGISTRY)/$(echo $IMAGE_NAME)" \
              .          
      - name: Login GITHUB container store
        run:  docker login ghcr.io -u $(echo $REGISTRY_USER) --password ${{ secrets.GITHUB_TOKEN }}

      - name: Push image to Github Container Registry
        run:  docker push $(echo $REGISTRY)/$(echo $IMAGE_NAME):${{ needs.version.outputs.GitVersion_SemVer }}

  
  release:
    name: release
    needs: [version, build]
    runs-on: ubuntu-latest
    permissions:
      contents: write
      packages: write
    env:
      myvar_fullSemVer: ${{ needs.version.outputs.SemVer }}

    steps:
      - name: Checkout code
        uses: actions/checkout@v3
        with:
          fetch-depth: 0


      - name: Setup dotnet
        uses: actions/setup-dotnet@v4
        with:
          dotnet-version: |
            8.0.x            
          
      - name: Create Change log
        id: changelog
        uses: TriPSs/conventional-changelog-action@v5.1.0
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          output-file: "false"
          preset: "conventionalcommits"
          skip-git-pull: true
          git-pull-method: --force
          skip-commit: true
          pre-release: true
          skip-version-file: true
          skip-tag: true
#          fallback-version: v${{ steps.version_step.outputs.semVer }} 
          fallback-version: v${{ needs.version.outputs.GitVersion_SemVer }}
          release-count: 0
          skip-on-empty: false
          tag-prefix: "v"
      #          create-summary: true
      #          skip-bump: true

      - name: Create tag
        uses: actions/github-script@v5
        continue-on-error: true
        with:
          script: |
                  github.rest.git.createRef({
                    owner: context.repo.owner,
                    repo: context.repo.repo,
                    ref: 'refs/tags/${{ needs.version.outputs.GitVersion_SemVer }}',
                    sha: context.sha
                  }) || true
                                    
      - name: Create GitHub Release
        uses: ncipollo/release-action@v1
        with:
          skipIfReleaseExists: true
          name: v${{ needs.version.outputs.GitVersion_SemVer }}
          tag: v${{ needs.version.outputs.GitVersion_SemVer }}
          body: "${{ steps.changelog.outputs.clean_changelog }} \r\n \r\n ### Related docker images \r\n * [${{ needs.build.outputs.DockerImage }}](${{ needs.build.outputs.DockerImageUrl }})"
          makeLatest: true
          

Explaining the code

line 6 -> 7: sets up release management based on branching
line 10 -> 14: Environment variables needed for the build

Important

Setting the above variables is enough to build the project. This script was used for deploying a dotnet core application. It uses GitVersion to automatically determine the version numbers upon every build. The method here is only compatible with dotnet core. For more information for other software check out: https://gitversion.net/docs/

Jobs

Version: determines the version number using GitVersion
Build: Builds the docker container using the Dockerfile and pushes the image to the container registry
Release: Creates a release using Semantic Versioning

You will be able to see the result in your repository

Gita

Gita is a wonderful tool that helps in managing multiple repositories as one. If you find yourself doing a lot of git actions through the command line. Check out this awesome tool.

GithubGita

Gita: a command-line tool to manage multiple git repos

This tool has two main features

display the status of multiple git repos such as branch, modification, commit message side by side
(batch) delegate git commands/aliases and shell commands on repos from any working directory

In this screenshot, the gita ll command displays the status of all repos. The gita remote dotfiles command translates to git remote -v for the dotfiles repo, even though we are not in the repo. The gita fetch command fetches from all repos and two of them have updates. To see the pre-defined commands, run gita -h or take a look at cmds.json. To add your own commands, see the customization section. To run arbitrary git command, see the superman mode section. To run arbitrary shell command, see the shell mode section.

I also made a youtube video to demonstrate the common usages

The branch color distinguishes 5 situations between local and remote branches:

color	meaning
white	local has no remote
green	local is the same as remote
red	local has diverged from remote
purple	local is ahead of remote (good for push)
yellow	local is behind remote (good for merge)

The choice of purple for ahead and yellow for behind is motivated by blueshift and redshift, using green as baseline. You can change the color scheme using the gita color command. See the customization section.

The additional status symbols denote

symbol	meaning
`+`	staged changes
`*`	unstaged changes
`?`	untracked files/folders
`$`	stashed contents

The bookkeeping sub-commands are

gita add <repo-path(s)> [-g <groupname>]: add repo(s) to gita, optionally into an existing group
gita add -a <repo-parent-path(s)>: add repo(s) in <repo-parent-path(s)> recursively and automatically generate hierarchical groups. See the customization section for more details.
gita add -b <bare-repo-path(s)>: add bare repo(s) to gita. See the customization section for more details on setting custom worktree.
gita add -r <repo-parent-path(s)>: add repo(s) in <repo-parent-path(s)> recursively
gita clear: remove all groups and repos
gita clone <URL>: clone repo from URL at current working directory
gita clone <URL> -C <directory>: change to directory and then clone repo
gita clone -f <config-file>: clone repos in config-file (generated by gita freeze) to current directory.
gita clone -p -f <config-file>: clone repos in config-file to prescribed paths.
gita context: context sub-command
- gita context: show current context
- gita context <group-name>: set context to group-name, all operations then only apply to repos in this group
- gita context auto: set context automatically according to the current working directory
- gita context none: remove context
gita color: color sub-command
- gita color [ll]: Show available colors and the current coloring scheme
- gita color reset: Reset to the default coloring scheme
- gita color set <situation> <color>: Use the specified color for the local-remote situation
gita flags: flags sub-command
- gita flags set <repo-name> <flags>: add custom flags to repo
- gita flags [ll]: display repos with custom flags
gita freeze: print information of all repos such as URL, name, and path. Use with gita clone.
gita group: group sub-command
- gita group add <repo-name(s)> -n <group-name>: add repo(s) to a new or existing group
- gita group [ll]: display existing groups with repos
- gita group ls: display existing group names
- gita group rename <group-name> <new-name>: change group name
- gita group rm <group-name(s)>: delete group(s)
- gita group rmrepo <repo-name(s)> -n <group-name>: remove repo(s) from existing group
gita info: info sub-command
- gita info [ll]: display the used and unused information items
- gita info add <info-item>: enable information item
- gita info rm <info-item>: disable information item
gita ll: display the status of all repos
gita ll <group-name>: display the status of repos in a group
gita ll -g: display the repo summaries by groups
gita ls: display the names of all repos
gita ls <repo-name>: display the absolute path of one repo
gita rename <repo-name> <new-name>: rename a repo
gita rm <repo-name(s)>: remove repo(s) from gita (won’t remove files on disk)
gita -v: display gita version

The git delegating sub-commands are of two formats

gita <sub-command> [repo-name(s) or group-name(s)]: optional repo or group input, and no input means all repos.
gita <sub-command> <repo-name(s) or groups-name(s)>: required repo name(s) or group name(s) input

They translate to git <sub-command> for the corresponding repos. By default, only fetch and pull take optional input. In other words, gita fetch and gita pull apply to all repos. To see the pre-defined sub-commands, run gita -h or take a look at cmds.json. To add your own sub-commands or override the default behaviors, see the customization section. To run arbitrary git command, see the superman mode section.

If more than one repos are specified, the git command runs asynchronously, with the exception of log, difftool and mergetool, which require non-trivial user input.

Repo configuration global is saved in $XDG_CONFIG_HOME/gita/repos.csv (most likely ~/.config/gita/repos.csv) or if you prefered at project configuration add environment variable GITA_PROJECT_HOME.

Installation

To install the latest version, run

pip3 install -U gita

If you prefer development mode, download the source code and run

pip3 install -e <gita-source-folder>

In either case, calling gita in terminal may not work, then put the following line in the .bashrc file.

alias gita="python3 -m gita"

Windows users may need to enable the ANSI escape sequence in terminal for the branch color to work. See this stackoverflow post for details.

Auto-completion

You can download the generated auto-completion file in the following locations for your specific shell. Alternatively, if you have installed argcomplete on your system, you can also directly run eval "$(register-python-argcomplete gita -s SHELL)" (e.g. SHELL as bash/zsh) in your dotfile.

Bash

Download .gita-completion.bash and source it in shell.

Zsh

There are 2 options :

.gita-completion.zsh. Use the help of gita command to display options. It uses the bash completion system for zsh. Add autoload -U +X bashcompinit && bashcompinit in .zshrc and source the zsh file
_gita. Completion more Zsh style. Copy it in a folder and add this folder path in FPATH variable. This completion file doesn’t take account to command from cmds.json

Fish

Download gita.fish and place it in ~/.config/fish/completions/

Superman mode

The superman mode delegates any git command or alias. Usage:

gita super [repo-name(s) or group-name(s)] <any-git-command-with-or-without-options>

Here repo-name(s) or group-name(s) are optional, and their absence means all repos. For example,

gita super checkout master puts all repos on the master branch
gita super frontend-repo backend-repo commit -am 'implement a new feature' executes git commit -am 'implement a new feature' for frontend-repo and backend-repo

Shell mode

The shell mode delegates any shell command. Usage:

gita shell [repo-name(s) or group-name(s)] <any-shell-command>

Here repo-name(s) or group-name(s) are optional, and their absence means all repos. For example,

gita shell ll lists contents for all repos
gita shell repo1 repo2 mkdir docs create a new directory docs in repo1 and repo2
gita shell "git describe --abbrev=0 --tags | xargs git checkout": check out the latest tag for all repos

Customization

define repo group and context

When the project contains several independent but related repos, we can define a group and execute gita command on this group. For example,

gita group add repo1 repo2 -n my-group
gita ll my-group
gita pull my-group

To save more typing, one can set a group as context, then any gita command is scoped to the group

gita context my-group
gita ll
gita pull

The most useful context maybe auto. In this mode, the context is automatically determined from the current working directory (CWD): the context is the group whose member repo’s path contains CWD. To set it, run

gita context auto

To remove the context, run

gita context none

It is also possible to recursively add repos within a directory and generate hierarchical groups automatically. For example, running

gita add -a src

on the following folder structure

src
├── project1
│   ├── repo1
│   └── repo2
├── repo3
├── project2
│   ├── repo4
│   └── repo5
└── repo6

gives rise to 3 groups:

src:repo1,repo2,repo3,repo4,repo5,repo6
src-project1:repo1,repo2
src-project2:repo4,repo5

add user-defined sub-command using json file

Custom delegating sub-commands can be defined in $XDG_CONFIG_HOME/gita/cmds.json (most likely ~/.config/gita/cmds.json) And they shadow the default ones if name collisions exist.

Default delegating sub-commands are defined in cmds.json. For example, gita stat <repo-name(s)> is registered as

1
2
3
4


"stat":{
  "cmd": "git diff --stat",
  "help": "show edit statistics"
}

which executes git diff --stat for the specified repo(s).

To disable asynchronous execution, set disable_async to be true. See the difftool example:

1
2
3
4
5


"difftool":{
  "cmd": "git difftool",
  "disable_async": true,
  "help": "show differences using a tool"
}

If you want a custom command to behave like gita fetch, i.e., to apply to all repos when no repo is specified, set allow_all to be true. For example, the following snippet creates a new command gita comaster [repo-name(s)] with optional repo name input.

1
2
3
4
5


"comaster":{
  "cmd": "checkout master",
  "allow_all": true,
  "help": "checkout the master branch"
}

Any command that runs in the superman mode mode or the shell mode can be defined in this json format. For example, the following command runs in shell mode and fetches only the current branch from upstream.

1
2
3
4
5
6


"fetchcrt":{
  "cmd": "git rev-parse --abbrev-ref HEAD | xargs git fetch --prune upstream",
  "allow_all": true,
  "shell": true,
  "help": "fetch current branch only"
}

customize the local/remote relationship coloring displayed by the `gita ll` command

You can see the default color scheme and the available colors via gita color. To change the color coding, use gita color set <situation> <color>. The configuration is saved in $XDG_CONFIG_HOME/gita/color.csv.

customize information displayed by the `gita ll` command

You can customize the information displayed by gita ll. The used and unused information items are shown with gita info, and the configuration is saved in $XDG_CONFIG_HOME/gita/info.csv.

For example, the default setting corresponds to

branch,commit_msg,commit_time

Here branch includes both branch name and status. The status symbols are similar to the ones used in spaceship-prompt.

To customize these symbols, add a file in $XDG_CONFIG_HOME/gita/symbols.csv. The default settings corresponds to

dirty,staged,untracked,local_ahead,remote_ahead,diverged,in_sync,no_remote
*,+,?,↑,↓,⇕,,∅

Only the symbols to be overridden need to be defined. You can search unicode symbols here.

customize git command flags

One can set custom flags to run git commands. For example, with

gita flags set my-repo --git-dir=`gita ls dotfiles` --work-tree=$HOME

any git command/alias triggered from gita on dotfiles will use these flags. Note that the flags are applied immediately after git. For example, gita st dotfiles translates to

git --git-dir=$HOME/somefolder --work-tree=$HOME status

running from the dotfiles directory.

This feature was originally added to deal with bare repo dotfiles.

Requirements

Gita requires Python 3.6 or higher, due to the use of f-string and asyncio module.

Under the hood, gita uses subprocess to run git commands/aliases. Thus the installed git version may matter. I have git 1.8.3.1, 2.17.2, and 2.20.1 on my machines, and their results agree.

Tips

effect	shell command
enter `<repo>` directory	cd `gita ls <repo>`
delete repos in `<group>`	`gita group ll <group> \| xargs gita rm`

Contributing

To contribute, you can

report/fix bugs
request/implement features
star/recommend this project

Read this article if you have never contribute code to open source project before.

Chat room is available on

To run tests locally, simply pytest in the source code folder. Note that context should be set as none. More implementation details are in design.md. A step-by-step guide to reproduce this project is here.

You can also sponsor me on GitHub. Any amount is appreciated!

Other multi-repo tools

I haven’t tried them but I heard good things about them.

Projects

Trading
Trading BotFor this project we will be using Freqtrade. An open source trading bot. With some very nice features, including backtesting, parameter optimization and strategy definitions. We can run it locally in a docker container. FreqtradeDocumentation Running the software To run the file we need a Dockerfile and a docker-compose.yml file. Dockerfile 1 2 3 4 FROM freqtradeorg/freqtrade:stable AS base WORKDIR /freqtrade EXPOSE 8080 COPY --chown=ftuser:ftuser . . Docker compose file 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 --- services: freqtrade: image: freqtradeorg/freqtrade:stable restart: unless-stopped container_name: freqtrade volumes: - ".
- Trading Bot
  For this project we will be using Freqtrade. An open source trading bot. With some very nice features, including backtesting, parameter optimization and strategy definitions. We can run it locally in a docker container. FreqtradeDocumentation Running the software To run the file we need a Dockerfile and a docker-compose.yml file. Dockerfile 1 2 3 4 FROM freqtradeorg/freqtrade:stable AS base WORKDIR /freqtrade EXPOSE 8080 COPY --chown=ftuser:ftuser . . Docker compose file 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 --- services: freqtrade: image: freqtradeorg/freqtrade:stable restart: unless-stopped container_name: freqtrade volumes: - ".
HRM
ComponentsComponents form the basis of our HRM system. Each component represents compensation and benefits an employee can receive as part of their salary package. This document describes what components are and how they are used throughout the entire system. As you will see components relate closely to policies, the only difference being that components are based on law, sectoral and institutional rules. They are defined by law and therefor informatively enforce those laws.
- Components
  Components form the basis of our HRM system. Each component represents compensation and benefits an employee can receive as part of their salary package. This document describes what components are and how they are used throughout the entire system. As you will see components relate closely to policies, the only difference being that components are based on law, sectoral and institutional rules. They are defined by law and therefor informatively enforce those laws.
- Policies
  Policies is a concept that drives the application. It is a new paradigm focussed on achieving truly dynamic applications. It is different from usual concepts for applications. Although the reference architecture is one that is widely known and is worked on by each and every organization. The basis for the policy system can be found in the security realm. Consider 2 types of applications: Data driven: Data is highest good, we have input and need to convert it to output Process driven: A process defines how an object moves through its different states, the data in that object becomes secondary Here we define a new way to approach a dynamic system.
- Client provisioning
  Client provisioning is the process of seting up and deploying a container for a new client. In this project we will focus on self-service setup. The idea is that a client can set up new environments and has access to them via single sign on. Process CamundaClient provisioning The process needs to differentiate between a new environment and an existing one. The steps to create or update an environment differ.
- Dynamic routes
  Dynamic routes are used to load layouts based on a structure in url format. These simplify the work needed to setup a page and allow for easy communication between dynamic components. Requirements Fixed structure Clean URL compliance wiki Unlimited length Component parameter support Pass information up/down the component tree Support commands Fixed structure Consider this url as an example: https://my-tool.org/employees/details/id/aaa-aaa-aaa/account/bbb-bbb-bbb/transaction/ccc-ccc-ccc The url has the following parts: host https://my-tool.org: this denotes the application url.

Trading

Trading Bot
For this project we will be using Freqtrade. An open source trading bot. With some very nice features, including backtesting, parameter optimization and strategy definitions. We can run it locally in a docker container. FreqtradeDocumentation Running the software To run the file we need a Dockerfile and a docker-compose.yml file. Dockerfile 1 2 3 4 FROM freqtradeorg/freqtrade:stable AS base WORKDIR /freqtrade EXPOSE 8080 COPY --chown=ftuser:ftuser . . Docker compose file 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 --- services: freqtrade: image: freqtradeorg/freqtrade:stable restart: unless-stopped container_name: freqtrade volumes: - ".

Trading Bot

For this project we will be using Freqtrade. An open source trading bot. With some very nice features, including backtesting, parameter optimization and strategy definitions. We can run it locally in a docker container.

FreqtradeDocumentation

Running the software

To run the file we need a Dockerfile and a docker-compose.yml file.

Dockerfile

1
2
3
4


FROM freqtradeorg/freqtrade:stable AS base
WORKDIR /freqtrade
EXPOSE 8080
COPY --chown=ftuser:ftuser . .

Docker compose file

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30


---
services:
  freqtrade:
    image: freqtradeorg/freqtrade:stable
    restart: unless-stopped
    container_name: freqtrade
    volumes:
      - "./user_data:/freqtrade/user_data"
    ports:
      - "0.0.0.0:8080:8080"
    command: >
      trade
      --logfile /freqtrade/user_data/logs/freqtrade.log
      --db-url sqlite:////freqtrade/user_data/tradesv3.sqlite
      --config /freqtrade/user_data/config-all-space.json
      --strategy NASOSv4      
  freqtrade2:
    image: freqtradeorg/freqtrade:stable
    restart: unless-stopped
    container_name: freqtrade2
    volumes:
      - "./user_data:/freqtrade/user_data"
    ports:
      - "0.0.0.0:8081:8080"
    command: >
      trade
      --logfile /freqtrade/user_data/logs/freqtrade.log
      --db-url sqlite:////freqtrade/user_data/tradesv3.sqlite
      --config /freqtrade/user_data/config-all-space.json
      --strategy ElliotV8_original_ichiv3      

Config file

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109


{
    "max_open_trades": 10,
    "stake_currency": "USDT",
    "stake_amount": "unlimited",
    "tradable_balance_ratio": 0.99,
    "fiat_display_currency": "EUR",
    "dry_run": true,
    "dry_run_wallet": 1000,
    "cancel_open_orders_on_exit": false,
    "trading_mode": "spot",
    "margin_mode": "",
    "dataformat_ohlcv": "json",
    "dataformat_trades": "json",
    "unfilledtimeout": {
        "entry": 10,
        "exit": 10,
        "exit_timeout_count": 0,
        "unit": "minutes"
    },
    "entry_pricing": {
        "price_side": "same",
        "use_order_book": true,
        "order_book_top": 1,
        "price_last_balance": 0.0,
        "check_depth_of_market": {
            "enabled": false,
            "bids_to_ask_delta": 1
        }
    },
    "exit_pricing":{
        "price_side": "same",
        "use_order_book": true,
        "order_book_top": 1
    },
    "exchange": {
        "name": "kucoin",
        "key": "<your_kucoin_key>",
        "secret": "<your_kucoin_secret>",
        "password": "<your_kucoin_password>",
        "ccxt_config": {},
        "ccxt_async_config": {},
        "pair_whitelist": [
            "NEAR/USDT",
            "RENDER/USDT",
            "INJ/USDT",
            "GRT/USDT",
            "AKT/USDT",
            "AIOZ/USDT",
            "FET/USDT",
            "HNT/USDT",
            "JASMY/USDT",
            "HAI/USDT",
            "DOGS/USDT",
            "XRP/USDT",
            "WIF/USDT",
            "DOGE/USDT",
            "LTC/USDT",
            "TON/USDT",
            "LINK/USDT",
            "SHIB/USDT",
            "FLOKI/USDT",
            "DOT/USDT",
            "ADA/USDT",
            "KAS/USDT",
            "ICP/USDT",
            "XMR/USDT",
            "SOL/USDT",
            "APT/USDT",
            "ETC/USDT",
            "XLM/USDT",
            "STX/USDT",
            "SUI/USDT",
            "CRO/USDT",
            "IMX/USDT",
            "FIL/USDT",
            "MNT/USDT"
        ],
        "pair_blacklist": [
        ]
    },
    "pairlists": [
        {"method": "StaticPairList"}
    ],
    "telegram": {
        "enabled": false,
        "token": "",
        "chat_id": ""
    },
    "api_server": {
        "enabled": true,
        "listen_ip_address": "0.0.0.0",
        "listen_port": 8080,
        "verbosity": "error",
        "enable_openapi": false,
        "jwt_secret_key": "<your_jwt_key>",
        "ws_token": "<your_token>",
        "CORS_origins": [
            "http://localhost:8080"
        ],
        "username": "freqtrader",
        "password": "<your_password>"
    },
    "bot_name": "freqtrade",
    "initial_state": "running",
    "force_entry_enable": false,
    "internals": {
        "process_throttle_secs": 5
    }
}

Important

Don’t forget to update the following data in the config file:

Exchange name
Exchange key
Exchange secret
Exchange password
API server JWT secret key
API server WS token
API username
API password

You can run the application with docker-compose up -d. It will launch the containers on your system.

To access the UI you can visit https://localhost:8080. You log in with the api username and api password you defined in the config file.

If you renamed the config file, you need to update the command in the docker-compose.yml file.

Running commands

Backtesting

1

docker-compose run freqtrade backtesting --export trades --stake-amount 100 --strategy-list BBRSIOptimizedStrategy VerhaertHedgeStrategy -i 15m

Hyperopt

1

    docker-compose run freqtrade hyperopt --hyperopt-loss SharpeHyperOptLoss --strategy VerhaertHedgeStrategy --spaces roi stoploss trades buy sell trailing -e 2500 -i 15m --timerange 20240501- --analyze-per-epoch --stake-amount 100 --max-open-trades 10 --min-trades 35

Download data

1

docker-compose run freqtrade download-data --exchange kucoin -t 1m 5m 15m 1h 4h 1d --erase --timerange 20240101-

Strategies

Strategies are used to determine how the bot takes trades. Lucky for us, a lot of strategies can be found on online. A nice resource is: https://www.dutchalgotrading.com/strategy-league/ with many thanks to Dutch Algotrading for providing this resource. He tested a lot of strategies and created a list of the most successful strategies.

Other strategies can be found on the Freqtrade github itself: https://github.com/freqtrade/freqtrade-strategies.

All we need to do is add them to the user_data/strategies folder. I’ll post some here for you to play with:

NASOSv4

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355


# for live trailing_stop = False and use_custom_stoploss = True
# for backtest trailing_stop = True and use_custom_stoploss = False

# --- Do not remove these libs ---
# --- Do not remove these libs ---
from logging import FATAL
from freqtrade.strategy.interface import IStrategy
from typing import Dict, List
from functools import reduce
from pandas import DataFrame
# --------------------------------
import talib.abstract as ta
import numpy as np
import freqtrade.vendor.qtpylib.indicators as qtpylib
import datetime
from technical.util import resample_to_interval, resampled_merge
from datetime import datetime, timedelta
from freqtrade.persistence import Trade
from freqtrade.strategy import stoploss_from_open, merge_informative_pair, DecimalParameter, IntParameter, CategoricalParameter
import technical.indicators as ftt

# @Rallipanos
# @pluxury

# Buy hyperspace params:
buy_params = {
    "base_nb_candles_buy": 8,
    "ewo_high": 2.403,
    "ewo_high_2": -5.585,
    "ewo_low": -14.378,
    "lookback_candles": 3,
    "low_offset": 0.984,
    "low_offset_2": 0.942,
    "profit_threshold": 1.008,
    "rsi_buy": 72
}

# Sell hyperspace params:
sell_params = {
    "base_nb_candles_sell": 16,
    "high_offset": 1.084,
    "high_offset_2": 1.401,
    "pHSL": -0.15,
    "pPF_1": 0.016,
    "pPF_2": 0.024,
    "pSL_1": 0.014,
    "pSL_2": 0.022
}


def EWO(dataframe, ema_length=5, ema2_length=35):
    df = dataframe.copy()
    ema1 = ta.EMA(df, timeperiod=ema_length)
    ema2 = ta.EMA(df, timeperiod=ema2_length)
    emadif = (ema1 - ema2) / df['low'] * 100
    return emadif


class NASOSv4(IStrategy):
    INTERFACE_VERSION = 2

    # ROI table:
    minimal_roi = {
        # "0": 0.283,
        # "40": 0.086,
        # "99": 0.036,
        "0": 10
    }

    # Stoploss:
    stoploss = -0.15

    # SMAOffset
    base_nb_candles_buy = IntParameter(
        2, 20, default=buy_params['base_nb_candles_buy'], space='buy', optimize=True)
    base_nb_candles_sell = IntParameter(
        2, 25, default=sell_params['base_nb_candles_sell'], space='sell', optimize=True)
    low_offset = DecimalParameter(
        0.9, 0.99, default=buy_params['low_offset'], space='buy', optimize=False)
    low_offset_2 = DecimalParameter(
        0.9, 0.99, default=buy_params['low_offset_2'], space='buy', optimize=False)
    high_offset = DecimalParameter(
        0.95, 1.1, default=sell_params['high_offset'], space='sell', optimize=True)
    high_offset_2 = DecimalParameter(
        0.99, 1.5, default=sell_params['high_offset_2'], space='sell', optimize=True)

    # Protection
    fast_ewo = 50
    slow_ewo = 200

    lookback_candles = IntParameter(
        1, 24, default=buy_params['lookback_candles'], space='buy', optimize=True)

    profit_threshold = DecimalParameter(1.0, 1.03,
                                        default=buy_params['profit_threshold'], space='buy', optimize=True)

    ewo_low = DecimalParameter(-20.0, -8.0,
                               default=buy_params['ewo_low'], space='buy', optimize=False)
    ewo_high = DecimalParameter(
        2.0, 12.0, default=buy_params['ewo_high'], space='buy', optimize=False)

    ewo_high_2 = DecimalParameter(
        -6.0, 12.0, default=buy_params['ewo_high_2'], space='buy', optimize=False)

    rsi_buy = IntParameter(50, 100, default=buy_params['rsi_buy'], space='buy', optimize=False)

    # trailing stoploss hyperopt parameters
    # hard stoploss profit
    pHSL = DecimalParameter(-0.200, -0.040, default=-0.15, decimals=3,
                            space='sell', optimize=False, load=True)
    # profit threshold 1, trigger point, SL_1 is used
    pPF_1 = DecimalParameter(0.008, 0.020, default=0.016, decimals=3,
                             space='sell', optimize=False, load=True)
    pSL_1 = DecimalParameter(0.008, 0.020, default=0.014, decimals=3,
                             space='sell', optimize=False, load=True)

    # profit threshold 2, SL_2 is used
    pPF_2 = DecimalParameter(0.040, 0.100, default=0.024, decimals=3,
                             space='sell', optimize=False, load=True)
    pSL_2 = DecimalParameter(0.020, 0.070, default=0.022, decimals=3,
                             space='sell', optimize=False, load=True)

    # Trailing stop:
    trailing_stop = True
    trailing_stop_positive = 0.001
    trailing_stop_positive_offset = 0.016
    trailing_only_offset_is_reached = True

    # Sell signal
    use_sell_signal = True
    sell_profit_only = False
    sell_profit_offset = 0.01
    ignore_roi_if_buy_signal = False

    # Optional order time in force.
    order_time_in_force = {
        'buy': 'gtc',
        'sell': 'ioc'
    }

    # Optimal timeframe for the strategy
    timeframe = '15m'
    inf_1h = '1h'

    process_only_new_candles = True
    startup_candle_count = 20
    use_custom_stoploss = False

    plot_config = {
        'main_plot': {
            'ma_buy': {'color': 'orange'},
            'ma_sell': {'color': 'orange'},
        },
    }

    slippage_protection = {
        'retries': 3,
        'max_slippage': -0.02
    }

    # Custom Trailing Stoploss by Perkmeister

    def custom_stoploss(self, pair: str, trade: 'Trade', current_time: datetime,
                        current_rate: float, current_profit: float, **kwargs) -> float:

        # # hard stoploss profit
        HSL = self.pHSL.value
        PF_1 = self.pPF_1.value
        SL_1 = self.pSL_1.value
        PF_2 = self.pPF_2.value
        SL_2 = self.pSL_2.value

        # For profits between PF_1 and PF_2 the stoploss (sl_profit) used is linearly interpolated
        # between the values of SL_1 and SL_2. For all profits above PL_2 the sl_profit value
        # rises linearly with current profit, for profits below PF_1 the hard stoploss profit is used.

        if (current_profit > PF_2):
            sl_profit = SL_2 + (current_profit - PF_2)
        elif (current_profit > PF_1):
            sl_profit = SL_1 + ((current_profit - PF_1)*(SL_2 - SL_1)/(PF_2 - PF_1))
        else:
            sl_profit = HSL

        # if current_profit < 0.001 and current_time - timedelta(minutes=600) > trade.open_date_utc:
        #     return -0.005

        return stoploss_from_open(sl_profit, current_profit)

    def confirm_trade_exit(self, pair: str, trade: Trade, order_type: str, amount: float,
                           rate: float, time_in_force: str, sell_reason: str,
                           current_time: datetime, **kwargs) -> bool:

        dataframe, _ = self.dp.get_analyzed_dataframe(pair, self.timeframe)
        last_candle = dataframe.iloc[-1]

        if (last_candle is not None):
            if (sell_reason in ['sell_signal']):
                if (last_candle['hma_50']*1.149 > last_candle['ema_100']) and (last_candle['close'] < last_candle['ema_100']*0.951):  # *1.2
                    return False

        # slippage
        try:
            state = self.slippage_protection['__pair_retries']
        except KeyError:
            state = self.slippage_protection['__pair_retries'] = {}

        candle = dataframe.iloc[-1].squeeze()

        slippage = (rate / candle['close']) - 1
        if slippage < self.slippage_protection['max_slippage']:
            pair_retries = state.get(pair, 0)
            if pair_retries < self.slippage_protection['retries']:
                state[pair] = pair_retries + 1
                return False

        state[pair] = 0

        return True

    def informative_pairs(self):
        pairs = self.dp.current_whitelist()
        informative_pairs = [(pair, '1h') for pair in pairs]
        return informative_pairs

    def informative_1h_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
        assert self.dp, "DataProvider is required for multiple timeframes."
        # Get the informative pair
        informative_1h = self.dp.get_pair_dataframe(pair=metadata['pair'], timeframe=self.inf_1h)
        # EMA
        # informative_1h['ema_50'] = ta.EMA(informative_1h, timeperiod=50)
        # informative_1h['ema_200'] = ta.EMA(informative_1h, timeperiod=200)
        # # RSI
        # informative_1h['rsi'] = ta.RSI(informative_1h, timeperiod=14)

        # bollinger = qtpylib.bollinger_bands(qtpylib.typical_price(dataframe), window=20, stds=2)
        # informative_1h['bb_lowerband'] = bollinger['lower']
        # informative_1h['bb_middleband'] = bollinger['mid']
        # informative_1h['bb_upperband'] = bollinger['upper']

        return informative_1h

    def normal_tf_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:

        # Calculate all ma_buy values
        for val in self.base_nb_candles_buy.range:
            dataframe[f'ma_buy_{val}'] = ta.EMA(dataframe, timeperiod=val)

        # Calculate all ma_sell values
        for val in self.base_nb_candles_sell.range:
            dataframe[f'ma_sell_{val}'] = ta.EMA(dataframe, timeperiod=val)

        dataframe['hma_50'] = qtpylib.hull_moving_average(dataframe['close'], window=50)
        dataframe['ema_100'] = ta.EMA(dataframe, timeperiod=100)

        dataframe['sma_9'] = ta.SMA(dataframe, timeperiod=9)
        # Elliot
        dataframe['EWO'] = EWO(dataframe, self.fast_ewo, self.slow_ewo)

        # RSI
        dataframe['rsi'] = ta.RSI(dataframe, timeperiod=14)
        dataframe['rsi_fast'] = ta.RSI(dataframe, timeperiod=4)
        dataframe['rsi_slow'] = ta.RSI(dataframe, timeperiod=20)

        return dataframe

    def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
        informative_1h = self.informative_1h_indicators(dataframe, metadata)
        dataframe = merge_informative_pair(
            dataframe, informative_1h, self.timeframe, self.inf_1h, ffill=True)

        # The indicators for the normal (5m) timeframe
        dataframe = self.normal_tf_indicators(dataframe, metadata)

        return dataframe

    def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:

        dont_buy_conditions = []

        dont_buy_conditions.append(
            (
                # don't buy if there isn't 3% profit to be made
                (dataframe['close_1h'].rolling(self.lookback_candles.value).max()
                 < (dataframe['close'] * self.profit_threshold.value))
            )
        )

        dataframe.loc[
            (
                (dataframe['rsi_fast'] < 35) &
                (dataframe['close'] < (dataframe[f'ma_buy_{self.base_nb_candles_buy.value}'] * self.low_offset.value)) &
                (dataframe['EWO'] > self.ewo_high.value) &
                (dataframe['rsi'] < self.rsi_buy.value) &
                (dataframe['volume'] > 0) &
                (dataframe['close'] < (
                    dataframe[f'ma_sell_{self.base_nb_candles_sell.value}'] * self.high_offset.value))
            ),
            ['buy', 'buy_tag']] = (1, 'ewo1')

        dataframe.loc[
            (
                (dataframe['rsi_fast'] < 35) &
                (dataframe['close'] < (dataframe[f'ma_buy_{self.base_nb_candles_buy.value}'] * self.low_offset_2.value)) &
                (dataframe['EWO'] > self.ewo_high_2.value) &
                (dataframe['rsi'] < self.rsi_buy.value) &
                (dataframe['volume'] > 0) &
                (dataframe['close'] < (dataframe[f'ma_sell_{self.base_nb_candles_sell.value}'] * self.high_offset.value)) &
                (dataframe['rsi'] < 25)
            ),
            ['buy', 'buy_tag']] = (1, 'ewo2')

        dataframe.loc[
            (
                (dataframe['rsi_fast'] < 35) &
                (dataframe['close'] < (dataframe[f'ma_buy_{self.base_nb_candles_buy.value}'] * self.low_offset.value)) &
                (dataframe['EWO'] < self.ewo_low.value) &
                (dataframe['volume'] > 0) &
                (dataframe['close'] < (
                    dataframe[f'ma_sell_{self.base_nb_candles_sell.value}'] * self.high_offset.value))
            ),
            ['buy', 'buy_tag']] = (1, 'ewolow')

        if dont_buy_conditions:
            for condition in dont_buy_conditions:
                dataframe.loc[condition, 'buy'] = 0

        return dataframe

    def populate_sell_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
        conditions = []

        conditions.append(
            ((dataframe['close'] > dataframe['sma_9']) &
                (dataframe['close'] > (dataframe[f'ma_sell_{self.base_nb_candles_sell.value}'] * self.high_offset_2.value)) &
                (dataframe['rsi'] > 50) &
                (dataframe['volume'] > 0) &
                (dataframe['rsi_fast'] > dataframe['rsi_slow'])
             )
            |
            (
                (dataframe['close'] < dataframe['hma_50']) &
                (dataframe['close'] > (dataframe[f'ma_sell_{self.base_nb_candles_sell.value}'] * self.high_offset.value)) &
                (dataframe['volume'] > 0) &
                (dataframe['rsi_fast'] > dataframe['rsi_slow'])
            )

        )

        if conditions:
            dataframe.loc[
                reduce(lambda x, y: x | y, conditions),
                'sell'
            ]=1

        return dataframe

NFI5MOHO_WIP

   1
   2
   3
   4
   5
   6
   7
   8
   9
  10
  11
  12
  13
  14
  15
  16
  17
  18
  19
  20
  21
  22
  23
  24
  25
  26
  27
  28
  29
  30
  31
  32
  33
  34
  35
  36
  37
  38
  39
  40
  41
  42
  43
  44
  45
  46
  47
  48
  49
  50
  51
  52
  53
  54
  55
  56
  57
  58
  59
  60
  61
  62
  63
  64
  65
  66
  67
  68
  69
  70
  71
  72
  73
  74
  75
  76
  77
  78
  79
  80
  81
  82
  83
  84
  85
  86
  87
  88
  89
  90
  91
  92
  93
  94
  95
  96
  97
  98
  99
 100
 101
 102
 103
 104
 105
 106
 107
 108
 109
 110
 111
 112
 113
 114
 115
 116
 117
 118
 119
 120
 121
 122
 123
 124
 125
 126
 127
 128
 129
 130
 131
 132
 133
 134
 135
 136
 137
 138
 139
 140
 141
 142
 143
 144
 145
 146
 147
 148
 149
 150
 151
 152
 153
 154
 155
 156
 157
 158
 159
 160
 161
 162
 163
 164
 165
 166
 167
 168
 169
 170
 171
 172
 173
 174
 175
 176
 177
 178
 179
 180
 181
 182
 183
 184
 185
 186
 187
 188
 189
 190
 191
 192
 193
 194
 195
 196
 197
 198
 199
 200
 201
 202
 203
 204
 205
 206
 207
 208
 209
 210
 211
 212
 213
 214
 215
 216
 217
 218
 219
 220
 221
 222
 223
 224
 225
 226
 227
 228
 229
 230
 231
 232
 233
 234
 235
 236
 237
 238
 239
 240
 241
 242
 243
 244
 245
 246
 247
 248
 249
 250
 251
 252
 253
 254
 255
 256
 257
 258
 259
 260
 261
 262
 263
 264
 265
 266
 267
 268
 269
 270
 271
 272
 273
 274
 275
 276
 277
 278
 279
 280
 281
 282
 283
 284
 285
 286
 287
 288
 289
 290
 291
 292
 293
 294
 295
 296
 297
 298
 299
 300
 301
 302
 303
 304
 305
 306
 307
 308
 309
 310
 311
 312
 313
 314
 315
 316
 317
 318
 319
 320
 321
 322
 323
 324
 325
 326
 327
 328
 329
 330
 331
 332
 333
 334
 335
 336
 337
 338
 339
 340
 341
 342
 343
 344
 345
 346
 347
 348
 349
 350
 351
 352
 353
 354
 355
 356
 357
 358
 359
 360
 361
 362
 363
 364
 365
 366
 367
 368
 369
 370
 371
 372
 373
 374
 375
 376
 377
 378
 379
 380
 381
 382
 383
 384
 385
 386
 387
 388
 389
 390
 391
 392
 393
 394
 395
 396
 397
 398
 399
 400
 401
 402
 403
 404
 405
 406
 407
 408
 409
 410
 411
 412
 413
 414
 415
 416
 417
 418
 419
 420
 421
 422
 423
 424
 425
 426
 427
 428
 429
 430
 431
 432
 433
 434
 435
 436
 437
 438
 439
 440
 441
 442
 443
 444
 445
 446
 447
 448
 449
 450
 451
 452
 453
 454
 455
 456
 457
 458
 459
 460
 461
 462
 463
 464
 465
 466
 467
 468
 469
 470
 471
 472
 473
 474
 475
 476
 477
 478
 479
 480
 481
 482
 483
 484
 485
 486
 487
 488
 489
 490
 491
 492
 493
 494
 495
 496
 497
 498
 499
 500
 501
 502
 503
 504
 505
 506
 507
 508
 509
 510
 511
 512
 513
 514
 515
 516
 517
 518
 519
 520
 521
 522
 523
 524
 525
 526
 527
 528
 529
 530
 531
 532
 533
 534
 535
 536
 537
 538
 539
 540
 541
 542
 543
 544
 545
 546
 547
 548
 549
 550
 551
 552
 553
 554
 555
 556
 557
 558
 559
 560
 561
 562
 563
 564
 565
 566
 567
 568
 569
 570
 571
 572
 573
 574
 575
 576
 577
 578
 579
 580
 581
 582
 583
 584
 585
 586
 587
 588
 589
 590
 591
 592
 593
 594
 595
 596
 597
 598
 599
 600
 601
 602
 603
 604
 605
 606
 607
 608
 609
 610
 611
 612
 613
 614
 615
 616
 617
 618
 619
 620
 621
 622
 623
 624
 625
 626
 627
 628
 629
 630
 631
 632
 633
 634
 635
 636
 637
 638
 639
 640
 641
 642
 643
 644
 645
 646
 647
 648
 649
 650
 651
 652
 653
 654
 655
 656
 657
 658
 659
 660
 661
 662
 663
 664
 665
 666
 667
 668
 669
 670
 671
 672
 673
 674
 675
 676
 677
 678
 679
 680
 681
 682
 683
 684
 685
 686
 687
 688
 689
 690
 691
 692
 693
 694
 695
 696
 697
 698
 699
 700
 701
 702
 703
 704
 705
 706
 707
 708
 709
 710
 711
 712
 713
 714
 715
 716
 717
 718
 719
 720
 721
 722
 723
 724
 725
 726
 727
 728
 729
 730
 731
 732
 733
 734
 735
 736
 737
 738
 739
 740
 741
 742
 743
 744
 745
 746
 747
 748
 749
 750
 751
 752
 753
 754
 755
 756
 757
 758
 759
 760
 761
 762
 763
 764
 765
 766
 767
 768
 769
 770
 771
 772
 773
 774
 775
 776
 777
 778
 779
 780
 781
 782
 783
 784
 785
 786
 787
 788
 789
 790
 791
 792
 793
 794
 795
 796
 797
 798
 799
 800
 801
 802
 803
 804
 805
 806
 807
 808
 809
 810
 811
 812
 813
 814
 815
 816
 817
 818
 819
 820
 821
 822
 823
 824
 825
 826
 827
 828
 829
 830
 831
 832
 833
 834
 835
 836
 837
 838
 839
 840
 841
 842
 843
 844
 845
 846
 847
 848
 849
 850
 851
 852
 853
 854
 855
 856
 857
 858
 859
 860
 861
 862
 863
 864
 865
 866
 867
 868
 869
 870
 871
 872
 873
 874
 875
 876
 877
 878
 879
 880
 881
 882
 883
 884
 885
 886
 887
 888
 889
 890
 891
 892
 893
 894
 895
 896
 897
 898
 899
 900
 901
 902
 903
 904
 905
 906
 907
 908
 909
 910
 911
 912
 913
 914
 915
 916
 917
 918
 919
 920
 921
 922
 923
 924
 925
 926
 927
 928
 929
 930
 931
 932
 933
 934
 935
 936
 937
 938
 939
 940
 941
 942
 943
 944
 945
 946
 947
 948
 949
 950
 951
 952
 953
 954
 955
 956
 957
 958
 959
 960
 961
 962
 963
 964
 965
 966
 967
 968
 969
 970
 971
 972
 973
 974
 975
 976
 977
 978
 979
 980
 981
 982
 983
 984
 985
 986
 987
 988
 989
 990
 991
 992
 993
 994
 995
 996
 997
 998
 999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188


import freqtrade.vendor.qtpylib.indicators as qtpylib
import numpy as np
import talib.abstract as ta
from freqtrade.strategy.interface import IStrategy
from freqtrade.strategy import (merge_informative_pair,
                                DecimalParameter, IntParameter, CategoricalParameter)
from pandas import DataFrame
from functools import reduce
from freqtrade.persistence import Trade
from datetime import datetime


###########################################################################################################
##                NostalgiaForInfinityV5 by iterativ                                                     ##
##                                                                                                       ##
##    Strategy for Freqtrade https://github.com/freqtrade/freqtrade                                      ##
##                                                                                                       ##
###########################################################################################################
##               GENERAL RECOMMENDATIONS                                                                 ##
##                                                                                                       ##
##   For optimal performance, suggested to use between 4 and 6 open trades, with unlimited stake.        ##
##   A pairlist with 40 to 80 pairs. Volume pairlist works well.                                         ##
##   Prefer stable coin (USDT, BUSDT etc) pairs, instead of BTC or ETH pairs.                            ##
##   Highly recommended to blacklist leveraged tokens (*BULL, *BEAR, *UP, *DOWN etc).                    ##
##   Ensure that you don't override any variables in you config.json. Especially                         ##
##   the timeframe (must be 5m).                                                                         ##
##     use_sell_signal must set to true (or not set at all).                                             ##
##     sell_profit_only must set to false (or not set at all).                                           ##
##     ignore_roi_if_buy_signal must set to true (or not set at all).                                    ##
##                                                                                                       ##
###########################################################################################################
##               DONATIONS                                                                               ##
##                                                                                                       ##
##   Absolutely not required. However, will be accepted as a token of appreciation.                      ##
##                                                                                                       ##
##   BTC: bc1qvflsvddkmxh7eqhc4jyu5z5k6xcw3ay8jl49sk                                                     ##
##   ETH (ERC20): 0x83D3cFb8001BDC5d2211cBeBB8cB3461E5f7Ec91                                             ##
##   BEP20/BSC (ETH, BNB, ...): 0x86A0B21a20b39d16424B7c8003E4A7e12d78ABEe                               ##
##                                                                                                       ##
###########################################################################################################

# 20210624
# NostalgiaForInfinityV5 + MultiOffsetLamboV0 + Hyper-optimized some parameters.

# I hope you do enough testing before proceeding.
# Thank you to those who created these strategies.

class NFI5MOHO_WIP(IStrategy):
    INTERFACE_VERSION = 2

    # Optional order type mapping.
    order_types = {
        'buy': 'limit',
        'sell': 'limit',
        'trailing_stop_loss': 'limit',
        'stoploss': 'limit',
        'stoploss_on_exchange': False
    }

    #############################################################

    buy_params = {
        #############
        # Enable/Disable conditions
        "buy_condition_1_enable": True,
        "buy_condition_2_enable": True,
        "buy_condition_3_enable": True,
        "buy_condition_4_enable": True,
        "buy_condition_5_enable": True,
        "buy_condition_6_enable": True,
        "buy_condition_7_enable": True,
        "buy_condition_8_enable": True,
        "buy_condition_9_enable": True,
        "buy_condition_10_enable": True,
        "buy_condition_11_enable": True,
        "buy_condition_12_enable": True,
        "buy_condition_13_enable": True,
        "buy_condition_14_enable": True,
        "buy_condition_15_enable": True,
        "buy_condition_16_enable": True,
        "buy_condition_17_enable": True,
        "buy_condition_18_enable": True,
        "buy_condition_19_enable": True,
        "buy_condition_20_enable": True,
        "buy_condition_21_enable": True,
        # Hyperopt
        # Multi Offset
	"""
	"base_nb_candles_buy": 42,
        "buy_chop_min_19": 29.3,
        "buy_rsi_1h_min_19": 52.4,
        "ewo_high": 5.262,
        "ewo_low": -8.164,
        "low_offset_ema": 0.984,
        "low_offset_kama": 0.919,
        "low_offset_sma": 0.97,
        "low_offset_t3": 0.904,
        "low_offset_trima": 0.984,
	"""
	"base_nb_candles_buy": 72,
        "buy_chop_min_19": 58.2,
        "buy_rsi_1h_min_19": 65.3,
        "ewo_high": 3.319,
        "ewo_low": -11.101,
        "low_offset_ema": 0.929,
        "low_offset_kama": 0.972,
        "low_offset_sma": 0.955,
        "low_offset_t3": 0.975,
        "low_offset_trima": 0.949,
    }

    sell_params = {
        #############
        # Enable/Disable conditions
        "sell_condition_1_enable": True,
        "sell_condition_2_enable": True,
        "sell_condition_3_enable": True,
        "sell_condition_4_enable": True,
        "sell_condition_5_enable": True,
        "sell_condition_6_enable": True,
        "sell_condition_7_enable": True,
        "sell_condition_8_enable": True,
        #############
        # Hyperopt
        # Multi Offset
        "base_nb_candles_sell": 34,
        "high_offset_ema": 1.047,
        "high_offset_kama": 1.07,
        "high_offset_sma": 1.051,
        "high_offset_t3": 0.999,
        "high_offset_trima": 1.096,
    }

    # ROI table:
    minimal_roi = {
        "0": 0.111,
        "13": 0.048,
        "50": 0.015,
        "61": 0.01
    }

    stoploss = -0.99

    # Multi Offset
    base_nb_candles_buy = IntParameter(
        5, 80, default=20, load=True, space='buy', optimize=True)
    base_nb_candles_sell = IntParameter(
        5, 80, default=20, load=True, space='sell', optimize=True)
    low_offset_sma = DecimalParameter(
        0.9, 0.99, default=0.958, load=True, space='buy', optimize=True)
    high_offset_sma = DecimalParameter(
        0.99, 1.1, default=1.012, load=True, space='sell', optimize=True)
    low_offset_ema = DecimalParameter(
        0.9, 0.99, default=0.958, load=True, space='buy', optimize=True)
    high_offset_ema = DecimalParameter(
        0.99, 1.1, default=1.012, load=True, space='sell', optimize=True)
    low_offset_trima = DecimalParameter(
        0.9, 0.99, default=0.958, load=True, space='buy', optimize=True)
    high_offset_trima = DecimalParameter(
        0.99, 1.1, default=1.012, load=True, space='sell', optimize=True)
    low_offset_t3 = DecimalParameter(
        0.9, 0.99, default=0.958, load=True, space='buy', optimize=True)
    high_offset_t3 = DecimalParameter(
        0.99, 1.1, default=1.012, load=True, space='sell', optimize=True)
    low_offset_kama = DecimalParameter(
        0.9, 0.99, default=0.958, load=True, space='buy', optimize=True)
    high_offset_kama = DecimalParameter(
        0.99, 1.1, default=1.012, load=True, space='sell', optimize=True)

    # Protection
    ewo_low = DecimalParameter(
        -20.0, -8.0, default=-20.0, load=True, space='buy', optimize=True)
    ewo_high = DecimalParameter(
        2.0, 12.0, default=6.0, load=True, space='buy', optimize=True)
    fast_ewo = IntParameter(
        10, 50, default=50, load=True, space='buy', optimize=False)
    slow_ewo = IntParameter(
        100, 200, default=200, load=True, space='buy', optimize=False)

    # MA list
    ma_types = ['sma', 'ema', 'trima', 't3', 'kama']
    ma_map = {
        'sma': {
            'low_offset': low_offset_sma.value,
            'high_offset': high_offset_sma.value,
            'calculate': ta.SMA
        },
        'ema': {
            'low_offset': low_offset_ema.value,
            'high_offset': high_offset_ema.value,
            'calculate': ta.EMA
        },
        'trima': {
            'low_offset': low_offset_trima.value,
            'high_offset': high_offset_trima.value,
            'calculate': ta.TRIMA
        },
        't3': {
            'low_offset': low_offset_t3.value,
            'high_offset': high_offset_t3.value,
            'calculate': ta.T3
        },
        'kama': {
            'low_offset': low_offset_kama.value,
            'high_offset': high_offset_kama.value,
            'calculate': ta.KAMA
        }
    }

    # Trailing stoploss (not used)
    trailing_stop = False
    trailing_only_offset_is_reached = True
    trailing_stop_positive = 0.01
    trailing_stop_positive_offset = 0.03

    use_custom_stoploss = False

    # Optimal timeframe for the strategy.
    timeframe = '5m'
    inf_1h = '1h'

    # Run "populate_indicators()" only for new candle.
    process_only_new_candles = True

    # These values can be overridden in the "ask_strategy" section in the config.
    use_sell_signal = True
    sell_profit_only = False
    ignore_roi_if_buy_signal = True

    # Number of candles the strategy requires before producing valid signals
    startup_candle_count: int = 300

    # plot config
    plot_config = {
        'main_plot': {
            'ma_offset_buy': {'color': 'orange'},
            'ma_offset_sell': {'color': 'orange'},
        },
    }

    #############################################################

    buy_condition_1_enable = CategoricalParameter([True, False], default=True, space='buy', optimize=False, load=True)
    buy_condition_2_enable = CategoricalParameter([True, False], default=True, space='buy', optimize=False, load=True)
    buy_condition_3_enable = CategoricalParameter([True, False], default=True, space='buy', optimize=False, load=True)
    buy_condition_4_enable = CategoricalParameter([True, False], default=True, space='buy', optimize=False, load=True)
    buy_condition_5_enable = CategoricalParameter([True, False], default=True, space='buy', optimize=False, load=True)
    buy_condition_6_enable = CategoricalParameter([True, False], default=True, space='buy', optimize=False, load=True)
    buy_condition_7_enable = CategoricalParameter([True, False], default=True, space='buy', optimize=False, load=True)
    buy_condition_8_enable = CategoricalParameter([True, False], default=True, space='buy', optimize=False, load=True)
    buy_condition_9_enable = CategoricalParameter([True, False], default=True, space='buy', optimize=False, load=True)
    buy_condition_10_enable = CategoricalParameter([True, False], default=True, space='buy', optimize=False, load=True)
    buy_condition_11_enable = CategoricalParameter([True, False], default=True, space='buy', optimize=False, load=True)
    buy_condition_12_enable = CategoricalParameter([True, False], default=True, space='buy', optimize=False, load=True)
    buy_condition_13_enable = CategoricalParameter([True, False], default=True, space='buy', optimize=False, load=True)
    buy_condition_14_enable = CategoricalParameter([True, False], default=True, space='buy', optimize=False, load=True)
    buy_condition_15_enable = CategoricalParameter([True, False], default=True, space='buy', optimize=False, load=True)
    buy_condition_16_enable = CategoricalParameter([True, False], default=True, space='buy', optimize=False, load=True)
    buy_condition_17_enable = CategoricalParameter([True, False], default=True, space='buy', optimize=False, load=True)
    buy_condition_18_enable = CategoricalParameter([True, False], default=True, space='buy', optimize=False, load=True)
    buy_condition_19_enable = CategoricalParameter([True, False], default=True, space='buy', optimize=False, load=True)
    buy_condition_20_enable = CategoricalParameter([True, False], default=True, space='buy', optimize=False, load=True)
    buy_condition_21_enable = CategoricalParameter([True, False], default=True, space='buy', optimize=False, load=True)

    # Normal dips
    buy_dip_threshold_1 = DecimalParameter(0.001, 0.05, default=0.02, space='buy', decimals=3, optimize=False, load=True)
    buy_dip_threshold_2 = DecimalParameter(0.01, 0.2, default=0.14, space='buy', decimals=3, optimize=False, load=True)
    buy_dip_threshold_3 = DecimalParameter(0.05, 0.4, default=0.32, space='buy', decimals=3, optimize=False, load=True)
    buy_dip_threshold_4 = DecimalParameter(0.2, 0.5, default=0.5, space='buy', decimals=3, optimize=False, load=True)
    # Strict dips
    buy_dip_threshold_5 = DecimalParameter(0.001, 0.05, default=0.015, space='buy', decimals=3, optimize=False, load=True)
    buy_dip_threshold_6 = DecimalParameter(0.01, 0.2, default=0.06, space='buy', decimals=3, optimize=False, load=True)
    buy_dip_threshold_7 = DecimalParameter(0.05, 0.4, default=0.24, space='buy', decimals=3, optimize=False, load=True)
    buy_dip_threshold_8 = DecimalParameter(0.2, 0.5, default=0.4, space='buy', decimals=3, optimize=False, load=True)
    # Loose dips
    buy_dip_threshold_9 = DecimalParameter(0.001, 0.05, default=0.026, space='buy', decimals=3, optimize=False, load=True)
    buy_dip_threshold_10 = DecimalParameter(0.01, 0.2, default=0.24, space='buy', decimals=3, optimize=False, load=True)
    buy_dip_threshold_11 = DecimalParameter(0.05, 0.4, default=0.42, space='buy', decimals=3, optimize=False, load=True)
    buy_dip_threshold_12 = DecimalParameter(0.2, 0.5, default=0.66, space='buy', decimals=3, optimize=False, load=True)

    # 24 hours
    buy_pump_pull_threshold_1 = DecimalParameter(1.5, 3.0, default=1.75, space='buy', decimals=2, optimize=False, load=True)
    buy_pump_threshold_1 = DecimalParameter(0.4, 1.0, default=0.5, space='buy', decimals=3, optimize=False, load=True)
    # 36 hours
    buy_pump_pull_threshold_2 = DecimalParameter(1.5, 3.0, default=1.75, space='buy', decimals=2, optimize=False, load=True)
    buy_pump_threshold_2 = DecimalParameter(0.4, 1.0, default=0.56, space='buy', decimals=3, optimize=False, load=True)
    # 48 hours
    buy_pump_pull_threshold_3 = DecimalParameter(1.5, 3.0, default=1.75, space='buy', decimals=2, optimize=False, load=True)
    buy_pump_threshold_3 = DecimalParameter(0.4, 1.0, default=0.85, space='buy', decimals=3, optimize=False, load=True)

    # 24 hours strict
    buy_pump_pull_threshold_4 = DecimalParameter(1.5, 3.0, default=2.2, space='buy', decimals=2, optimize=False, load=True)
    buy_pump_threshold_4 = DecimalParameter(0.4, 1.0, default=0.4, space='buy', decimals=3, optimize=False, load=True)
    # 36 hours strict
    buy_pump_pull_threshold_5 = DecimalParameter(1.5, 3.0, default=2.0, space='buy', decimals=2, optimize=False, load=True)
    buy_pump_threshold_5 = DecimalParameter(0.4, 1.0, default=0.56, space='buy', decimals=3, optimize=False, load=True)
    # 48 hours strict
    buy_pump_pull_threshold_6 = DecimalParameter(1.5, 3.0, default=2.0, space='buy', decimals=2, optimize=False, load=True)
    buy_pump_threshold_6 = DecimalParameter(0.4, 1.0, default=0.68, space='buy', decimals=3, optimize=False, load=True)

    # 24 hours loose
    buy_pump_pull_threshold_7 = DecimalParameter(1.5, 3.0, default=1.7, space='buy', decimals=2, optimize=False, load=True)
    buy_pump_threshold_7 = DecimalParameter(0.4, 1.0, default=0.66, space='buy', decimals=3, optimize=False, load=True)
    # 36 hours loose
    buy_pump_pull_threshold_8 = DecimalParameter(1.5, 3.0, default=1.7, space='buy', decimals=2, optimize=False, load=True)
    buy_pump_threshold_8 = DecimalParameter(0.4, 1.0, default=0.7, space='buy', decimals=3, optimize=False, load=True)
    # 48 hours loose
    buy_pump_pull_threshold_9 = DecimalParameter(1.5, 3.0, default=1.4, space='buy', decimals=2, optimize=False, load=True)
    buy_pump_threshold_9 = DecimalParameter(0.4, 1.8, default=1.3, space='buy', decimals=3, optimize=False, load=True)

    buy_min_inc_1 = DecimalParameter(0.01, 0.05, default=0.022, space='buy', decimals=3, optimize=False, load=True)
    buy_rsi_1h_min_1 = DecimalParameter(25.0, 40.0, default=30.0, space='buy', decimals=1, optimize=False, load=True)
    buy_rsi_1h_max_1 = DecimalParameter(70.0, 90.0, default=84.0, space='buy', decimals=1, optimize=False, load=True)
    buy_rsi_1 = DecimalParameter(20.0, 40.0, default=36.0, space='buy', decimals=1, optimize=False, load=True)
    buy_mfi_1 = DecimalParameter(20.0, 40.0, default=26.0, space='buy', decimals=1, optimize=False, load=True)

    buy_volume_2 = DecimalParameter(1.0, 10.0, default=2.6, space='buy', decimals=1, optimize=False, load=True)
    buy_rsi_1h_min_2 = DecimalParameter(30.0, 40.0, default=32.0, space='buy', decimals=1, optimize=False, load=True)
    buy_rsi_1h_max_2 = DecimalParameter(70.0, 95.0, default=84.0, space='buy', decimals=1, optimize=False, load=True)
    buy_rsi_1h_diff_2 = DecimalParameter(30.0, 50.0, default=39.0, space='buy', decimals=1, optimize=False, load=True)
    buy_mfi_2 = DecimalParameter(30.0, 56.0, default=49.0, space='buy', decimals=1, optimize=False, load=True)
    buy_bb_offset_2 = DecimalParameter(0.97, 0.999, default=0.983, space='buy', decimals=3, optimize=False, load=True)

    buy_bb40_bbdelta_close_3 = DecimalParameter(0.005, 0.06, default=0.057, space='buy', optimize=False, load=True)
    buy_bb40_closedelta_close_3 = DecimalParameter(0.01, 0.03, default=0.023, space='buy', optimize=False, load=True)
    buy_bb40_tail_bbdelta_3 = DecimalParameter(0.15, 0.45, default=0.418, space='buy', optimize=False, load=True)
    buy_ema_rel_3 = DecimalParameter(0.97, 0.999, default=0.986, space='buy', decimals=3, optimize=False, load=True)

    buy_bb20_close_bblowerband_4 = DecimalParameter(0.96, 0.99, default=0.979, space='buy', optimize=False, load=True)
    buy_bb20_volume_4 = DecimalParameter(1.0, 20.0, default=10.0, space='buy', decimals=2, optimize=False, load=True)

    buy_ema_open_mult_5 = DecimalParameter(0.016, 0.03, default=0.019, space='buy', decimals=3, optimize=False, load=True)
    buy_bb_offset_5 = DecimalParameter(0.98, 1.0, default=0.999, space='buy', decimals=3, optimize=False, load=True)
    buy_ema_rel_5 = DecimalParameter(0.97, 0.999, default=0.982, space='buy', decimals=3, optimize=False, load=True)

    buy_ema_open_mult_6 = DecimalParameter(0.02, 0.03, default=0.025, space='buy', decimals=3, optimize=False, load=True)
    buy_bb_offset_6 = DecimalParameter(0.98, 0.999, default=0.984, space='buy', decimals=3, optimize=False, load=True)

    buy_volume_7 = DecimalParameter(1.0, 10.0, default=2.0, space='buy', decimals=1, optimize=False, load=True)
    buy_ema_open_mult_7 = DecimalParameter(0.02, 0.04, default=0.03, space='buy', decimals=3, optimize=False, load=True)
    buy_rsi_7 = DecimalParameter(24.0, 50.0, default=36.0, space='buy', decimals=1, optimize=False, load=True)
    buy_ema_rel_7 = DecimalParameter(0.97, 0.999, default=0.986, space='buy', decimals=3, optimize=False, load=True)

    buy_volume_8 = DecimalParameter(1.0, 6.0, default=2.0, space='buy', decimals=1, optimize=False, load=True)
    buy_rsi_8 = DecimalParameter(36.0, 40.0, default=20.0, space='buy', decimals=1, optimize=False, load=True)
    buy_tail_diff_8 = DecimalParameter(3.0, 10.0, default=3.5, space='buy', decimals=1, optimize=False, load=True)

    buy_volume_9 = DecimalParameter(1.0, 4.0, default=1.0, space='buy', decimals=2, optimize=False, load=True)
    buy_ma_offset_9 = DecimalParameter(0.94, 0.99, default=0.97, space='buy', decimals=3, optimize=False, load=True)
    buy_bb_offset_9 = DecimalParameter(0.97, 0.99, default=0.985, space='buy', decimals=3, optimize=False, load=True)
    buy_rsi_1h_min_9 = DecimalParameter(26.0, 40.0, default=30.0, space='buy', decimals=1, optimize=False, load=True)
    buy_rsi_1h_max_9 = DecimalParameter(70.0, 90.0, default=88.0, space='buy', decimals=1, optimize=False, load=True)
    buy_mfi_9 = DecimalParameter(36.0, 65.0, default=30.0, space='buy', decimals=1, optimize=False, load=True)

    buy_volume_10 = DecimalParameter(1.0, 8.0, default=2.4, space='buy', decimals=1, optimize=False, load=True)
    buy_ma_offset_10 = DecimalParameter(0.93, 0.97, default=0.944, space='buy', decimals=3, optimize=False, load=True)
    buy_bb_offset_10 = DecimalParameter(0.97, 0.99, default=0.994, space='buy', decimals=3, optimize=False, load=True)
    buy_rsi_1h_10 = DecimalParameter(20.0, 40.0, default=37.0, space='buy', decimals=1, optimize=False, load=True)

    buy_ma_offset_11 = DecimalParameter(0.93, 0.99, default=0.939, space='buy', decimals=3, optimize=False, load=True)
    buy_min_inc_11 = DecimalParameter(0.005, 0.05, default=0.022, space='buy', decimals=3, optimize=False, load=True)
    buy_rsi_1h_min_11 = DecimalParameter(40.0, 60.0, default=56.0, space='buy', decimals=1, optimize=False, load=True)
    buy_rsi_1h_max_11 = DecimalParameter(70.0, 90.0, default=84.0, space='buy', decimals=1, optimize=False, load=True)
    buy_rsi_11 = DecimalParameter(30.0, 48.0, default=48.0, space='buy', decimals=1, optimize=False, load=True)
    buy_mfi_11 = DecimalParameter(36.0, 56.0, default=38.0, space='buy', decimals=1, optimize=False, load=True)

    buy_volume_12 = DecimalParameter(1.0, 10.0, default=1.7, space='buy', decimals=1, optimize=False, load=True)
    buy_ma_offset_12 = DecimalParameter(0.93, 0.97, default=0.936, space='buy', decimals=3, optimize=False, load=True)
    buy_rsi_12 = DecimalParameter(26.0, 40.0, default=30.0, space='buy', decimals=1, optimize=False, load=True)
    buy_ewo_12 = DecimalParameter(2.0, 6.0, default=2.0, space='buy', decimals=1, optimize=False, load=True)

    buy_volume_13 = DecimalParameter(1.0, 10.0, default=1.6, space='buy', decimals=1, optimize=False, load=True)
    buy_ma_offset_13 = DecimalParameter(0.93, 0.98, default=0.978, space='buy', decimals=3, optimize=False, load=True)
    buy_ewo_13 = DecimalParameter(-14.0, -7.0, default=-10.4, space='buy', decimals=1, optimize=False, load=True)

    buy_volume_14 = DecimalParameter(1.0, 10.0, default=2.0, space='buy', decimals=1, optimize=False, load=True)
    buy_ema_open_mult_14 = DecimalParameter(0.01, 0.03, default=0.014, space='buy', decimals=3, optimize=False, load=True)
    buy_bb_offset_14 = DecimalParameter(0.98, 1.0, default=0.986, space='buy', decimals=3, optimize=False, load=True)
    buy_ma_offset_14 = DecimalParameter(0.93, 0.99, default=0.97, space='buy', decimals=3, optimize=False, load=True)

    buy_volume_15 = DecimalParameter(1.0, 10.0, default=2.0, space='buy', decimals=1, optimize=False, load=True)
    buy_ema_open_mult_15 = DecimalParameter(0.02, 0.04, default=0.018, space='buy', decimals=3, optimize=False, load=True)
    buy_ma_offset_15 = DecimalParameter(0.93, 0.99, default=0.954, space='buy', decimals=3, optimize=False, load=True)
    buy_rsi_15 = DecimalParameter(30.0, 50.0, default=28.0, space='buy', decimals=1, optimize=False, load=True)
    buy_ema_rel_15 = DecimalParameter(0.97, 0.999, default=0.988, space='buy', decimals=3, optimize=False, load=True)

    buy_volume_16 = DecimalParameter(1.0, 10.0, default=2.0, space='buy', decimals=1, optimize=False, load=True)
    buy_ma_offset_16 = DecimalParameter(0.93, 0.97, default=0.952, space='buy', decimals=3, optimize=False, load=True)
    buy_rsi_16 = DecimalParameter(26.0, 50.0, default=31.0, space='buy', decimals=1, optimize=False, load=True)
    buy_ewo_16 = DecimalParameter(4.0, 8.0, default=2.8, space='buy', decimals=1, optimize=False, load=True)

    buy_volume_17 = DecimalParameter(0.5, 8.0, default=2.0, space='buy', decimals=1, optimize=False, load=True)
    buy_ma_offset_17 = DecimalParameter(0.93, 0.98, default=0.958, space='buy', decimals=3, optimize=False, load=True)
    buy_ewo_17 = DecimalParameter(-18.0, -10.0, default=-12.0, space='buy', decimals=1, optimize=False, load=True)

    buy_volume_18 = DecimalParameter(1.0, 6.0, default=2.0, space='buy', decimals=1, optimize=False, load=True)
    buy_rsi_18 = DecimalParameter(16.0, 32.0, default=26.0, space='buy', decimals=1, optimize=False, load=True)
    buy_bb_offset_18 = DecimalParameter(0.98, 1.0, default=0.982, space='buy', decimals=3, optimize=False, load=True)

    buy_rsi_1h_min_19 = DecimalParameter(40.0, 70.0, default=50.0, space='buy', decimals=1, optimize=True, load=True)
    buy_chop_min_19 = DecimalParameter(20.0, 60.0, default=24.1, space='buy', decimals=1, optimize=True, load=True)

    buy_volume_20 = DecimalParameter(0.5, 6.0, default=1.2, space='buy', decimals=1, optimize=False, load=True)
    #buy_ema_rel_20 = DecimalParameter(0.97, 0.999, default=0.988, space='buy', decimals=3, optimize=False, load=True)
    buy_rsi_20 = DecimalParameter(20.0, 36.0, default=26.0, space='buy', decimals=1, optimize=False, load=True)
    buy_rsi_1h_20 = DecimalParameter(14.0, 30.0, default=20.0, space='buy', decimals=1, optimize=False, load=True)

    buy_volume_21 = DecimalParameter(0.5, 6.0, default=3.0, space='buy', decimals=1, optimize=False, load=True)
    #buy_ema_rel_21 = DecimalParameter(0.97, 0.999, default=0.988, space='buy', decimals=3, optimize=False, load=True)
    buy_rsi_21 = DecimalParameter(10.0, 28.0, default=23.0, space='buy', decimals=1, optimize=False, load=True)
    buy_rsi_1h_21 = DecimalParameter(18.0, 40.0, default=24.0, space='buy', decimals=1, optimize=False, load=True)

    # Sell

    sell_condition_1_enable = CategoricalParameter([True, False], default=True, space='sell', optimize=False, load=True)
    sell_condition_2_enable = CategoricalParameter([True, False], default=True, space='sell', optimize=False, load=True)
    sell_condition_3_enable = CategoricalParameter([True, False], default=True, space='sell', optimize=False, load=True)
    sell_condition_4_enable = CategoricalParameter([True, False], default=True, space='sell', optimize=False, load=True)
    sell_condition_5_enable = CategoricalParameter([True, False], default=True, space='sell', optimize=False, load=True)
    sell_condition_6_enable = CategoricalParameter([True, False], default=True, space='sell', optimize=False, load=True)
    sell_condition_7_enable = CategoricalParameter([True, False], default=True, space='sell', optimize=False, load=True)
    sell_condition_8_enable = CategoricalParameter([True, False], default=True, space='sell', optimize=False, load=True)

    sell_rsi_bb_1 = DecimalParameter(60.0, 80.0, default=79.5, space='sell', decimals=1, optimize=False, load=True)

    sell_rsi_bb_2 = DecimalParameter(72.0, 90.0, default=81, space='sell', decimals=1, optimize=False, load=True)

    sell_rsi_main_3 = DecimalParameter(77.0, 90.0, default=82, space='sell', decimals=1, optimize=False, load=True)

    sell_dual_rsi_rsi_4 = DecimalParameter(72.0, 84.0, default=73.4, space='sell', decimals=1, optimize=False, load=True)
    sell_dual_rsi_rsi_1h_4 = DecimalParameter(78.0, 92.0, default=79.6, space='sell', decimals=1, optimize=False, load=True)

    sell_ema_relative_5 = DecimalParameter(0.005, 0.05, default=0.024, space='sell', optimize=False, load=True)
    sell_rsi_diff_5 = DecimalParameter(0.0, 20.0, default=4.4, space='sell', optimize=False, load=True)

    sell_rsi_under_6 = DecimalParameter(72.0, 90.0, default=79.0, space='sell', decimals=1, optimize=False, load=True)

    sell_rsi_1h_7 = DecimalParameter(80.0, 95.0, default=81.7, space='sell', decimals=1, optimize=False, load=True)

    sell_bb_relative_8 = DecimalParameter(1.05, 1.3, default=1.1, space='sell', decimals=3, optimize=False, load=True)

    sell_custom_profit_0 = DecimalParameter(0.01, 0.1, default=0.01, space='sell', decimals=3, optimize=False, load=True)
    sell_custom_rsi_0 = DecimalParameter(30.0, 40.0, default=33.0, space='sell', decimals=3, optimize=False, load=True)
    sell_custom_profit_1 = DecimalParameter(0.01, 0.1, default=0.03, space='sell', decimals=3, optimize=False, load=True)
    sell_custom_rsi_1 = DecimalParameter(30.0, 50.0, default=38.0, space='sell', decimals=2, optimize=False, load=True)
    sell_custom_profit_2 = DecimalParameter(0.01, 0.1, default=0.05, space='sell', decimals=3, optimize=False, load=True)
    sell_custom_rsi_2 = DecimalParameter(34.0, 50.0, default=43.0, space='sell', decimals=2, optimize=False, load=True)
    sell_custom_profit_3 = DecimalParameter(0.06, 0.30, default=0.08, space='sell', decimals=3, optimize=False, load=True)
    sell_custom_rsi_3 = DecimalParameter(38.0, 55.0, default=48.0, space='sell', decimals=2, optimize=False, load=True)
    sell_custom_profit_4 = DecimalParameter(0.3, 0.6, default=0.25, space='sell', decimals=3, optimize=False, load=True)
    sell_custom_rsi_4 = DecimalParameter(40.0, 58.0, default=50.0, space='sell', decimals=2, optimize=False, load=True)

    sell_custom_under_profit_1 = DecimalParameter(0.01, 0.10, default=0.02, space='sell', decimals=3, optimize=False, load=True)
    sell_custom_under_rsi_1 = DecimalParameter(36.0, 60.0, default=56.0, space='sell', decimals=1, optimize=False, load=True)
    sell_custom_under_profit_2 = DecimalParameter(0.01, 0.10, default=0.04, space='sell', decimals=3, optimize=False, load=True)
    sell_custom_under_rsi_2 = DecimalParameter(46.0, 66.0, default=60.0, space='sell', decimals=1, optimize=False, load=True)
    sell_custom_under_profit_3 = DecimalParameter(0.01, 0.10, default=0.6, space='sell', decimals=3, optimize=False, load=True)
    sell_custom_under_rsi_3 = DecimalParameter(50.0, 68.0, default=62.0, space='sell', decimals=1, optimize=False, load=True)

    sell_custom_dec_profit_1 = DecimalParameter(0.01, 0.10, default=0.05, space='sell', decimals=3, optimize=False, load=True)
    sell_custom_dec_profit_2 = DecimalParameter(0.05, 0.2, default=0.07, space='sell', decimals=3, optimize=False, load=True)

    sell_trail_profit_min_1 = DecimalParameter(0.1, 0.25, default=0.15, space='sell', decimals=3, optimize=False, load=True)
    sell_trail_profit_max_1 = DecimalParameter(0.3, 0.5, default=0.46, space='sell', decimals=2, optimize=False, load=True)
    sell_trail_down_1 = DecimalParameter(0.04, 0.2, default=0.18, space='sell', decimals=3, optimize=False, load=True)

    sell_trail_profit_min_2 = DecimalParameter(0.01, 0.1, default=0.01, space='sell', decimals=3, optimize=False, load=True)
    sell_trail_profit_max_2 = DecimalParameter(0.08, 0.25, default=0.12, space='sell', decimals=2, optimize=False, load=True)
    sell_trail_down_2 = DecimalParameter(0.04, 0.2, default=0.14, space='sell', decimals=3, optimize=False, load=True)

    sell_trail_profit_min_3 = DecimalParameter(0.01, 0.1, default=0.05, space='sell', decimals=3, optimize=False, load=True)
    sell_trail_profit_max_3 = DecimalParameter(0.08, 0.16, default=0.1, space='sell', decimals=2, optimize=False, load=True)
    sell_trail_down_3 = DecimalParameter(0.01, 0.04, default=0.01, space='sell', decimals=3, optimize=False, load=True)

    sell_custom_profit_under_rel_1 = DecimalParameter(0.01, 0.04, default=0.024, space='sell', optimize=False, load=True)
    sell_custom_profit_under_rsi_diff_1 = DecimalParameter(0.0, 20.0, default=4.4, space='sell', optimize=False, load=True)

    sell_custom_stoploss_under_rel_1 = DecimalParameter(0.001, 0.02, default=0.004, space='sell', optimize=False, load=True)
    sell_custom_stoploss_under_rsi_diff_1 = DecimalParameter(0.0, 20.0, default=8.0, space='sell', optimize=False, load=True)

    #############################################################

    def get_ticker_indicator(self):
        return int(self.timeframe[:-1])


    def custom_sell(self, pair: str, trade: 'Trade', current_time: 'datetime', current_rate: float,
                    current_profit: float, **kwargs):
        dataframe, _ = self.dp.get_analyzed_dataframe(pair, self.timeframe)
        last_candle = dataframe.iloc[-1].squeeze()

        max_profit = ((trade.max_rate - trade.open_rate) / trade.open_rate)

        if (last_candle is not None):
            if (current_profit > self.sell_custom_profit_4.value) & (last_candle['rsi'] < self.sell_custom_rsi_4.value):
                return 'signal_profit_4'
            elif (current_profit > self.sell_custom_profit_3.value) & (last_candle['rsi'] < self.sell_custom_rsi_3.value):
                return 'signal_profit_3'
            elif (current_profit > self.sell_custom_profit_2.value) & (last_candle['rsi'] < self.sell_custom_rsi_2.value):
                return 'signal_profit_2'
            elif (current_profit > self.sell_custom_profit_1.value) & (last_candle['rsi'] < self.sell_custom_rsi_1.value):
                return 'signal_profit_1'
            elif (current_profit > self.sell_custom_profit_0.value) & (last_candle['rsi'] < self.sell_custom_rsi_0.value):
                return 'signal_profit_0'

            elif (current_profit > self.sell_custom_under_profit_1.value) & (last_candle['rsi'] < self.sell_custom_under_rsi_1.value) & (last_candle['close'] < last_candle['ema_200']):
                return 'signal_profit_u_1'
            elif (current_profit > self.sell_custom_under_profit_2.value) & (last_candle['rsi'] < self.sell_custom_under_rsi_2.value) & (last_candle['close'] < last_candle['ema_200']):
                return 'signal_profit_u_2'
            elif (current_profit > self.sell_custom_under_profit_3.value) & (last_candle['rsi'] < self.sell_custom_under_rsi_3.value) & (last_candle['close'] < last_candle['ema_200']):
                return 'signal_profit_u_3'

            elif (current_profit > self.sell_custom_dec_profit_1.value) & (last_candle['sma_200_dec']):
                return 'signal_profit_d_1'
            elif (current_profit > self.sell_custom_dec_profit_2.value) & (last_candle['close'] < last_candle['ema_100']):
                return 'signal_profit_d_2'

            elif (current_profit > self.sell_trail_profit_min_1.value) & (current_profit < self.sell_trail_profit_max_1.value) & (max_profit > (current_profit + self.sell_trail_down_1.value)):
                return 'signal_profit_t_1'
            elif (current_profit > self.sell_trail_profit_min_2.value) & (current_profit < self.sell_trail_profit_max_2.value) & (max_profit > (current_profit + self.sell_trail_down_2.value)):
                return 'signal_profit_t_2'

            elif (last_candle['close'] < last_candle['ema_200']) & (current_profit > self.sell_trail_profit_min_3.value) & (current_profit < self.sell_trail_profit_max_3.value) & (max_profit > (current_profit + self.sell_trail_down_3.value)):
                return 'signal_profit_u_t_1'

            elif (current_profit > 0.0) & (last_candle['close'] < last_candle['ema_200']) & (((last_candle['ema_200'] - last_candle['close']) / last_candle['close']) < self.sell_custom_profit_under_rel_1.value) & (last_candle['rsi'] > last_candle['rsi_1h'] + self.sell_custom_profit_under_rsi_diff_1.value):
                return 'signal_profit_u_e_1'

            elif (current_profit < -0.0) & (last_candle['close'] < last_candle['ema_200']) & (((last_candle['ema_200'] - last_candle['close']) / last_candle['close']) < self.sell_custom_stoploss_under_rel_1.value) & (last_candle['rsi'] > last_candle['rsi_1h'] + self.sell_custom_stoploss_under_rsi_diff_1.value):
                return 'signal_stoploss_u_1'

        return None

    def informative_pairs(self):
        # get access to all pairs available in whitelist.
        pairs = self.dp.current_whitelist()
        # Assign tf to each pair so they can be downloaded and cached for strategy.
        informative_pairs = [(pair, '1h') for pair in pairs]
        return informative_pairs

    def informative_1h_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
        assert self.dp, "DataProvider is required for multiple timeframes."
        # Get the informative pair
        informative_1h = self.dp.get_pair_dataframe(pair=metadata['pair'], timeframe=self.inf_1h)
        # EMA
        informative_1h['ema_15'] = ta.EMA(informative_1h, timeperiod=15)
        informative_1h['ema_50'] = ta.EMA(informative_1h, timeperiod=50)
        informative_1h['ema_100'] = ta.EMA(informative_1h, timeperiod=100)
        informative_1h['ema_200'] = ta.EMA(informative_1h, timeperiod=200)
        # SMA
        informative_1h['sma_200'] = ta.SMA(informative_1h, timeperiod=200)
        # RSI
        informative_1h['rsi'] = ta.RSI(informative_1h, timeperiod=14)
        # BB
        bollinger = qtpylib.bollinger_bands(qtpylib.typical_price(informative_1h), window=20, stds=2)
        informative_1h['bb_lowerband'] = bollinger['lower']
        informative_1h['bb_middleband'] = bollinger['mid']
        informative_1h['bb_upperband'] = bollinger['upper']
        # Pump protections
        informative_1h['safe_pump_24'] = ((((informative_1h['open'].rolling(24).max() - informative_1h['close'].rolling(24).min()) / informative_1h['close'].rolling(24).min()) < self.buy_pump_threshold_1.value) | (((informative_1h['open'].rolling(24).max() - informative_1h['close'].rolling(24).min()) / self.buy_pump_pull_threshold_1.value) > (informative_1h['close'] - informative_1h['close'].rolling(24).min())))
        informative_1h['safe_pump_36'] = ((((informative_1h['open'].rolling(36).max() - informative_1h['close'].rolling(36).min()) / informative_1h['close'].rolling(36).min()) < self.buy_pump_threshold_2.value) | (((informative_1h['open'].rolling(36).max() - informative_1h['close'].rolling(36).min()) / self.buy_pump_pull_threshold_2.value) > (informative_1h['close'] - informative_1h['close'].rolling(36).min())))
        informative_1h['safe_pump_48'] = ((((informative_1h['open'].rolling(48).max() - informative_1h['close'].rolling(48).min()) / informative_1h['close'].rolling(48).min()) < self.buy_pump_threshold_3.value) | (((informative_1h['open'].rolling(48).max() - informative_1h['close'].rolling(48).min()) / self.buy_pump_pull_threshold_3.value) > (informative_1h['close'] - informative_1h['close'].rolling(48).min())))

        informative_1h['safe_pump_24_strict'] = ((((informative_1h['open'].rolling(24).max() - informative_1h['close'].rolling(24).min()) / informative_1h['close'].rolling(24).min()) < self.buy_pump_threshold_4.value) | (((informative_1h['open'].rolling(24).max() - informative_1h['close'].rolling(24).min()) / self.buy_pump_pull_threshold_4.value) > (informative_1h['close'] - informative_1h['close'].rolling(24).min())))
        informative_1h['safe_pump_36_strict'] = ((((informative_1h['open'].rolling(36).max() - informative_1h['close'].rolling(36).min()) / informative_1h['close'].rolling(36).min()) < self.buy_pump_threshold_5.value) | (((informative_1h['open'].rolling(36).max() - informative_1h['close'].rolling(36).min()) / self.buy_pump_pull_threshold_5.value) > (informative_1h['close'] - informative_1h['close'].rolling(36).min())))
        informative_1h['safe_pump_48_strict'] = ((((informative_1h['open'].rolling(48).max() - informative_1h['close'].rolling(48).min()) / informative_1h['close'].rolling(48).min()) < self.buy_pump_threshold_6.value) | (((informative_1h['open'].rolling(48).max() - informative_1h['close'].rolling(48).min()) / self.buy_pump_pull_threshold_6.value) > (informative_1h['close'] - informative_1h['close'].rolling(48).min())))

        informative_1h['safe_pump_24_loose'] = ((((informative_1h['open'].rolling(24).max() - informative_1h['close'].rolling(24).min()) / informative_1h['close'].rolling(24).min()) < self.buy_pump_threshold_7.value) | (((informative_1h['open'].rolling(24).max() - informative_1h['close'].rolling(24).min()) / self.buy_pump_pull_threshold_7.value) > (informative_1h['close'] - informative_1h['close'].rolling(24).min())))
        informative_1h['safe_pump_36_loose'] = ((((informative_1h['open'].rolling(36).max() - informative_1h['close'].rolling(36).min()) / informative_1h['close'].rolling(36).min()) < self.buy_pump_threshold_8.value) | (((informative_1h['open'].rolling(36).max() - informative_1h['close'].rolling(36).min()) / self.buy_pump_pull_threshold_8.value) > (informative_1h['close'] - informative_1h['close'].rolling(36).min())))
        informative_1h['safe_pump_48_loose'] = ((((informative_1h['open'].rolling(48).max() - informative_1h['close'].rolling(48).min()) / informative_1h['close'].rolling(48).min()) < self.buy_pump_threshold_9.value) | (((informative_1h['open'].rolling(48).max() - informative_1h['close'].rolling(48).min()) / self.buy_pump_pull_threshold_9.value) > (informative_1h['close'] - informative_1h['close'].rolling(48).min())))

        return informative_1h

    def normal_tf_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
        # BB 40
        bb_40 = qtpylib.bollinger_bands(dataframe['close'], window=40, stds=2)
        dataframe['lower'] = bb_40['lower']
        dataframe['mid'] = bb_40['mid']
        dataframe['bbdelta'] = (bb_40['mid'] - dataframe['lower']).abs()
        dataframe['closedelta'] = (dataframe['close'] - dataframe['close'].shift()).abs()
        dataframe['tail'] = (dataframe['close'] - dataframe['low']).abs()

        # BB 20
        bollinger = qtpylib.bollinger_bands(qtpylib.typical_price(dataframe), window=20, stds=2)
        dataframe['bb_lowerband'] = bollinger['lower']
        dataframe['bb_middleband'] = bollinger['mid']
        dataframe['bb_upperband'] = bollinger['upper']

        # EMA 200
        dataframe['ema_12'] = ta.EMA(dataframe, timeperiod=12)
        dataframe['ema_20'] = ta.EMA(dataframe, timeperiod=20)
        dataframe['ema_26'] = ta.EMA(dataframe, timeperiod=26)
        dataframe['ema_50'] = ta.EMA(dataframe, timeperiod=50)
        dataframe['ema_100'] = ta.EMA(dataframe, timeperiod=100)
        dataframe['ema_200'] = ta.EMA(dataframe, timeperiod=200)

        # SMA
        dataframe['sma_5'] = ta.SMA(dataframe, timeperiod=5)
        dataframe['sma_30'] = ta.SMA(dataframe, timeperiod=30)
        dataframe['sma_200'] = ta.SMA(dataframe, timeperiod=200)

        dataframe['sma_200_dec'] = dataframe['sma_200'] < dataframe['sma_200'].shift(20)

        # MFI
        dataframe['mfi'] = ta.MFI(dataframe)

        # EWO
        dataframe['ewo'] = EWO(dataframe, self.fast_ewo.value, self.slow_ewo.value)

        # RSI
        dataframe['rsi'] = ta.RSI(dataframe, timeperiod=14)

        # Chopiness
        dataframe['chop']= qtpylib.chopiness(dataframe, 14)

        # Dip protection
        dataframe['safe_dips'] = ((((dataframe['open'] - dataframe['close']) / dataframe['close']) < self.buy_dip_threshold_1.value) &
                                  (((dataframe['open'].rolling(2).max() - dataframe['close']) / dataframe['close']) < self.buy_dip_threshold_2.value) &
                                  (((dataframe['open'].rolling(12).max() - dataframe['close']) / dataframe['close']) < self.buy_dip_threshold_3.value) &
                                  (((dataframe['open'].rolling(144).max() - dataframe['close']) / dataframe['close']) < self.buy_dip_threshold_4.value))

        dataframe['safe_dips_strict'] = ((((dataframe['open'] - dataframe['close']) / dataframe['close']) < self.buy_dip_threshold_5.value) &
                                  (((dataframe['open'].rolling(2).max() - dataframe['close']) / dataframe['close']) < self.buy_dip_threshold_6.value) &
                                  (((dataframe['open'].rolling(12).max() - dataframe['close']) / dataframe['close']) < self.buy_dip_threshold_7.value) &
                                  (((dataframe['open'].rolling(144).max() - dataframe['close']) / dataframe['close']) < self.buy_dip_threshold_8.value))

        dataframe['safe_dips_loose'] = ((((dataframe['open'] - dataframe['close']) / dataframe['close']) < self.buy_dip_threshold_9.value) &
                                  (((dataframe['open'].rolling(2).max() - dataframe['close']) / dataframe['close']) < self.buy_dip_threshold_10.value) &
                                  (((dataframe['open'].rolling(12).max() - dataframe['close']) / dataframe['close']) < self.buy_dip_threshold_11.value) &
                                  (((dataframe['open'].rolling(144).max() - dataframe['close']) / dataframe['close']) < self.buy_dip_threshold_12.value))

        # Volume
        dataframe['volume_mean_4'] = dataframe['volume'].rolling(4).mean().shift(1)
        dataframe['volume_mean_30'] = dataframe['volume'].rolling(30).mean()

        # Offset
        for i in self.ma_types:
            dataframe[f'{i}_offset_buy'] = self.ma_map[f'{i}']['calculate'](
                dataframe, self.base_nb_candles_buy.value) * \
                self.ma_map[f'{i}']['low_offset']
            dataframe[f'{i}_offset_sell'] = self.ma_map[f'{i}']['calculate'](
                dataframe, self.base_nb_candles_sell.value) * \
                self.ma_map[f'{i}']['high_offset']

        return dataframe

    def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
        # The indicators for the 1h informative timeframe
        informative_1h = self.informative_1h_indicators(dataframe, metadata)
        dataframe = merge_informative_pair(dataframe, informative_1h, self.timeframe, self.inf_1h, ffill=True)

        # The indicators for the normal (5m) timeframe
        dataframe = self.normal_tf_indicators(dataframe, metadata)

        return dataframe


    def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
        conditions = []

        conditions.append(
            (
                self.buy_condition_1_enable.value &

                (dataframe['ema_50_1h'] > dataframe['ema_200_1h']) &
                (dataframe['sma_200'] > dataframe['sma_200'].shift(50)) &

                (dataframe['safe_dips_strict']) &
                (dataframe['safe_pump_24_1h']) &

                (((dataframe['close'] - dataframe['open'].rolling(36).min()) / dataframe['open'].rolling(36).min()) > self.buy_min_inc_1.value) &
                (dataframe['rsi_1h'] > self.buy_rsi_1h_min_1.value) &
                (dataframe['rsi_1h'] < self.buy_rsi_1h_max_1.value) &
                (dataframe['rsi'] < self.buy_rsi_1.value) &
                (dataframe['mfi'] < self.buy_mfi_1.value) &

                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.buy_condition_2_enable.value &

                (dataframe['sma_200_1h'] > dataframe['sma_200_1h'].shift(50)) &

                (dataframe['safe_pump_24_strict_1h']) &

                (dataframe['volume_mean_4'] * self.buy_volume_2.value > dataframe['volume']) &

                #(dataframe['rsi_1h'] > self.buy_rsi_1h_min_2.value) &
                #(dataframe['rsi_1h'] < self.buy_rsi_1h_max_2.value) &
                (dataframe['rsi'] < dataframe['rsi_1h'] - self.buy_rsi_1h_diff_2.value) &
                (dataframe['mfi'] < self.buy_mfi_2.value) &
                (dataframe['close'] < (dataframe['bb_lowerband'] * self.buy_bb_offset_2.value)) &

                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.buy_condition_3_enable.value &

                (dataframe['close'] > (dataframe['ema_200_1h'] * self.buy_ema_rel_3.value)) &
                (dataframe['ema_100'] > dataframe['ema_200']) &
                (dataframe['ema_100_1h'] > dataframe['ema_200_1h']) &

                (dataframe['safe_pump_36_strict_1h']) &

                dataframe['lower'].shift().gt(0) &
                dataframe['bbdelta'].gt(dataframe['close'] * self.buy_bb40_bbdelta_close_3.value) &
                dataframe['closedelta'].gt(dataframe['close'] * self.buy_bb40_closedelta_close_3.value) &
                dataframe['tail'].lt(dataframe['bbdelta'] * self.buy_bb40_tail_bbdelta_3.value) &
                dataframe['close'].lt(dataframe['lower'].shift()) &
                dataframe['close'].le(dataframe['close'].shift()) &
                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.buy_condition_4_enable.value &

                (dataframe['ema_50_1h'] > dataframe['ema_200_1h']) &

                (dataframe['safe_dips_strict']) &
                (dataframe['safe_pump_24_1h']) &

                (dataframe['close'] < dataframe['ema_50']) &
                (dataframe['close'] < self.buy_bb20_close_bblowerband_4.value * dataframe['bb_lowerband']) &
                (dataframe['volume'] < (dataframe['volume_mean_30'].shift(1) * self.buy_bb20_volume_4.value))
            )
        )

        conditions.append(
            (
                self.buy_condition_5_enable.value &

                (dataframe['ema_100'] > dataframe['ema_200']) &
                (dataframe['close'] > (dataframe['ema_200_1h'] * self.buy_ema_rel_5.value)) &

                (dataframe['safe_dips']) &
                (dataframe['safe_pump_36_strict_1h']) &

                (dataframe['ema_26'] > dataframe['ema_12']) &
                ((dataframe['ema_26'] - dataframe['ema_12']) > (dataframe['open'] * self.buy_ema_open_mult_5.value)) &
                ((dataframe['ema_26'].shift() - dataframe['ema_12'].shift()) > (dataframe['open'] / 100)) &
                (dataframe['close'] < (dataframe['bb_lowerband'] * self.buy_bb_offset_5.value)) &

                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.buy_condition_6_enable.value &

                (dataframe['ema_100_1h'] > dataframe['ema_200_1h']) &

                (dataframe['safe_dips_loose']) &
                (dataframe['safe_pump_36_strict_1h']) &

                (dataframe['ema_26'] > dataframe['ema_12']) &
                ((dataframe['ema_26'] - dataframe['ema_12']) > (dataframe['open'] * self.buy_ema_open_mult_6.value)) &
                ((dataframe['ema_26'].shift() - dataframe['ema_12'].shift()) > (dataframe['open'] / 100)) &
                (dataframe['close'] < (dataframe['bb_lowerband'] * self.buy_bb_offset_6.value)) &

                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.buy_condition_7_enable.value &

                (dataframe['ema_100'] > dataframe['ema_200']) &
                (dataframe['ema_50_1h'] > dataframe['ema_200_1h']) &

                (dataframe['safe_dips_strict']) &

                (dataframe['volume'].rolling(4).mean() * self.buy_volume_7.value > dataframe['volume']) &

                (dataframe['ema_26'] > dataframe['ema_12']) &
                ((dataframe['ema_26'] - dataframe['ema_12']) > (dataframe['open'] * self.buy_ema_open_mult_7.value)) &
                ((dataframe['ema_26'].shift() - dataframe['ema_12'].shift()) > (dataframe['open'] / 100)) &
                (dataframe['rsi'] < self.buy_rsi_7.value) &

                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.buy_condition_8_enable.value &

                (dataframe['ema_50_1h'] > dataframe['ema_200_1h']) &

                (dataframe['safe_dips_loose']) &
                (dataframe['safe_pump_24_1h']) &

                (dataframe['rsi'] < self.buy_rsi_8.value) &
                (dataframe['volume'] > (dataframe['volume'].shift(1) * self.buy_volume_8.value)) &
                (dataframe['close'] > dataframe['open']) &
                ((dataframe['close'] - dataframe['low']) > ((dataframe['close'] - dataframe['open']) * self.buy_tail_diff_8.value)) &

                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.buy_condition_9_enable.value &

                (dataframe['ema_50'] > dataframe['ema_200']) &
                (dataframe['ema_100'] > dataframe['ema_200']) &

                (dataframe['safe_dips_strict']) &
                (dataframe['safe_pump_24_loose_1h']) &

                (dataframe['volume_mean_4'] * self.buy_volume_9.value > dataframe['volume']) &

                (dataframe['close'] < dataframe['ema_20'] * self.buy_ma_offset_9.value) &
                (dataframe['close'] < dataframe['bb_lowerband'] * self.buy_bb_offset_9.value) &
                (dataframe['rsi_1h'] > self.buy_rsi_1h_min_9.value) &
                (dataframe['rsi_1h'] < self.buy_rsi_1h_max_9.value) &
                (dataframe['mfi'] < self.buy_mfi_9.value) &
                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.buy_condition_10_enable.value &

                (dataframe['ema_50_1h'] > dataframe['ema_100_1h']) &
                (dataframe['sma_200_1h'] > dataframe['sma_200_1h'].shift(24)) &

                (dataframe['safe_dips_loose']) &
                (dataframe['safe_pump_24_loose_1h']) &

                ((dataframe['volume_mean_4'] * self.buy_volume_10.value) > dataframe['volume']) &

                (dataframe['close'] < dataframe['sma_30'] * self.buy_ma_offset_10.value) &
                (dataframe['close'] < dataframe['bb_lowerband'] * self.buy_bb_offset_10.value) &
                (dataframe['rsi_1h'] < self.buy_rsi_1h_10.value) &
                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.buy_condition_11_enable.value &

                (dataframe['ema_50_1h'] > dataframe['ema_100_1h']) &

                (dataframe['safe_dips_loose']) &
                (dataframe['safe_pump_24_loose_1h']) &
                (dataframe['safe_pump_36_1h']) &
                (dataframe['safe_pump_48_loose_1h']) &

                (((dataframe['close'] - dataframe['open'].rolling(36).min()) / dataframe['open'].rolling(36).min()) > self.buy_min_inc_11.value) &
                (dataframe['close'] < dataframe['sma_30'] * self.buy_ma_offset_11.value) &
                (dataframe['rsi_1h'] > self.buy_rsi_1h_min_11.value) &
                (dataframe['rsi_1h'] < self.buy_rsi_1h_max_11.value) &
                (dataframe['rsi'] < self.buy_rsi_11.value) &
                (dataframe['mfi'] < self.buy_mfi_11.value) &
                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.buy_condition_12_enable.value &

                (dataframe['sma_200_1h'] > dataframe['sma_200_1h'].shift(24)) &

                (dataframe['safe_dips_strict']) &
                (dataframe['safe_pump_24_1h']) &

                ((dataframe['volume_mean_4'] * self.buy_volume_12.value) > dataframe['volume']) &

                (dataframe['close'] < dataframe['sma_30'] * self.buy_ma_offset_12.value) &
                (dataframe['ewo'] > self.buy_ewo_12.value) &
                (dataframe['rsi'] < self.buy_rsi_12.value) &
                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.buy_condition_13_enable.value &

                (dataframe['ema_50_1h'] > dataframe['ema_100_1h']) &
                (dataframe['sma_200_1h'] > dataframe['sma_200_1h'].shift(24)) &

                (dataframe['safe_dips_strict']) &
                (dataframe['safe_pump_24_loose_1h']) &
                (dataframe['safe_pump_36_loose_1h']) &

                ((dataframe['volume_mean_4'] * self.buy_volume_13.value) > dataframe['volume']) &

                (dataframe['close'] < dataframe['sma_30'] * self.buy_ma_offset_13.value) &
                (dataframe['ewo'] < self.buy_ewo_13.value) &
                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.buy_condition_14_enable.value &

                (dataframe['sma_200'] > dataframe['sma_200'].shift(30)) &
                (dataframe['sma_200_1h'] > dataframe['sma_200_1h'].shift(50)) &

                (dataframe['safe_dips_loose']) &
                (dataframe['safe_pump_24_1h']) &

                (dataframe['volume_mean_4'] * self.buy_volume_14.value > dataframe['volume']) &

                (dataframe['ema_26'] > dataframe['ema_12']) &
                ((dataframe['ema_26'] - dataframe['ema_12']) > (dataframe['open'] * self.buy_ema_open_mult_14.value)) &
                ((dataframe['ema_26'].shift() - dataframe['ema_12'].shift()) > (dataframe['open'] / 100)) &
                (dataframe['close'] < (dataframe['bb_lowerband'] * self.buy_bb_offset_14.value)) &
                (dataframe['close'] < dataframe['ema_20'] * self.buy_ma_offset_14.value) &

                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.buy_condition_15_enable.value &

                (dataframe['close'] > dataframe['ema_200_1h'] * self.buy_ema_rel_15.value) &
                (dataframe['ema_50_1h'] > dataframe['ema_200_1h']) &

                (dataframe['safe_dips']) &
                (dataframe['safe_pump_36_strict_1h']) &

                (dataframe['ema_26'] > dataframe['ema_12']) &
                ((dataframe['ema_26'] - dataframe['ema_12']) > (dataframe['open'] * self.buy_ema_open_mult_15.value)) &
                ((dataframe['ema_26'].shift() - dataframe['ema_12'].shift()) > (dataframe['open'] / 100)) &
                (dataframe['rsi'] < self.buy_rsi_15.value) &
                (dataframe['close'] < dataframe['ema_20'] * self.buy_ma_offset_15.value) &

                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.buy_condition_16_enable.value &

                (dataframe['ema_50_1h'] > dataframe['ema_200_1h']) &

                (dataframe['safe_dips_strict']) &
                (dataframe['safe_pump_24_strict_1h']) &

                ((dataframe['volume_mean_4'] * self.buy_volume_16.value) > dataframe['volume']) &

                (dataframe['close'] < dataframe['ema_20'] * self.buy_ma_offset_16.value) &
                (dataframe['ewo'] > self.buy_ewo_16.value) &
                (dataframe['rsi'] < self.buy_rsi_16.value) &
                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.buy_condition_17_enable.value &

                (dataframe['safe_dips_strict']) &
                (dataframe['safe_pump_24_loose_1h']) &

                ((dataframe['volume_mean_4'] * self.buy_volume_17.value) > dataframe['volume']) &

                (dataframe['close'] < dataframe['ema_20'] * self.buy_ma_offset_17.value) &
                (dataframe['ewo'] < self.buy_ewo_17.value) &

                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.buy_condition_18_enable.value &

                (dataframe['close'] > dataframe['ema_200_1h']) &
                (dataframe['ema_100'] > dataframe['ema_200']) &
                (dataframe['ema_50_1h'] > dataframe['ema_200_1h']) &
                (dataframe['sma_200'] > dataframe['sma_200'].shift(20)) &
                (dataframe['sma_200'] > dataframe['sma_200'].shift(44)) &
                (dataframe['sma_200_1h'] > dataframe['sma_200_1h'].shift(36)) &
                (dataframe['sma_200_1h'] > dataframe['sma_200_1h'].shift(72)) &

                (dataframe['safe_dips']) &
                (dataframe['safe_pump_24_strict_1h']) &

                ((dataframe['volume_mean_4'] * self.buy_volume_18.value) > dataframe['volume']) &

                (dataframe['rsi'] < self.buy_rsi_18.value) &
                (dataframe['close'] < (dataframe['bb_lowerband'] * self.buy_bb_offset_18.value)) &
                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.buy_condition_19_enable.value &

                (dataframe['ema_100_1h'] > dataframe['ema_200_1h']) &

                (dataframe['sma_200'] > dataframe['sma_200'].shift(36)) &
                (dataframe['ema_50_1h'] > dataframe['ema_200_1h']) &

                (dataframe['safe_dips']) &
                (dataframe['safe_pump_24_1h']) &

                (dataframe['close'].shift(1) > dataframe['ema_100_1h']) &
                (dataframe['low'] < dataframe['ema_100_1h']) &
                (dataframe['close'] > dataframe['ema_100_1h']) &
                (dataframe['rsi_1h'] > self.buy_rsi_1h_min_19.value) &
                (dataframe['chop'] < self.buy_chop_min_19.value) &
                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.buy_condition_20_enable.value &

                (dataframe['ema_50_1h'] > dataframe['ema_200_1h']) &

                (dataframe['safe_dips']) &
                (dataframe['safe_pump_24_loose_1h']) &

                ((dataframe['volume_mean_4'] * self.buy_volume_20.value) > dataframe['volume']) &

                (dataframe['rsi'] < self.buy_rsi_20.value) &
                (dataframe['rsi_1h'] < self.buy_rsi_1h_20.value) &

                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.buy_condition_21_enable.value &

                (dataframe['ema_50_1h'] > dataframe['ema_200_1h']) &

                (dataframe['safe_dips_strict']) &

                ((dataframe['volume_mean_4'] * self.buy_volume_21.value) > dataframe['volume']) &

                (dataframe['rsi'] < self.buy_rsi_21.value) &
                (dataframe['rsi_1h'] < self.buy_rsi_1h_21.value) &

                (dataframe['volume'] > 0)
            )
        )

        for i in self.ma_types:
            conditions.append(
                (
                    dataframe['close'] < dataframe[f'{i}_offset_buy']) &
                (
                    (dataframe['ewo'] < self.ewo_low.value) |
                    (dataframe['ewo'] > self.ewo_high.value)
                ) &
                (dataframe['volume'] > 0)
        )

        if conditions:
            dataframe.loc[
                reduce(lambda x, y: x | y, conditions),
                'buy'
            ] = 1

        return dataframe

    def populate_sell_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
        conditions = []

        conditions.append(
            (
                self.sell_condition_1_enable.value &

                (dataframe['rsi'] > self.sell_rsi_bb_1.value) &
                (dataframe['close'] > dataframe['bb_upperband']) &
                (dataframe['close'].shift(1) > dataframe['bb_upperband'].shift(1)) &
                (dataframe['close'].shift(2) > dataframe['bb_upperband'].shift(2)) &
                (dataframe['close'].shift(3) > dataframe['bb_upperband'].shift(3)) &
                (dataframe['close'].shift(4) > dataframe['bb_upperband'].shift(4)) &
                (dataframe['close'].shift(5) > dataframe['bb_upperband'].shift(5)) &
                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.sell_condition_2_enable.value &

                (dataframe['rsi'] > self.sell_rsi_bb_2.value) &
                (dataframe['close'] > dataframe['bb_upperband']) &
                (dataframe['close'].shift(1) > dataframe['bb_upperband'].shift(1)) &
                (dataframe['close'].shift(2) > dataframe['bb_upperband'].shift(2)) &
                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.sell_condition_3_enable.value &

                (dataframe['rsi'] > self.sell_rsi_main_3.value) &
                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.sell_condition_4_enable.value &

                (dataframe['rsi'] > self.sell_dual_rsi_rsi_4.value) &
                (dataframe['rsi_1h'] > self.sell_dual_rsi_rsi_1h_4.value) &
                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.sell_condition_6_enable.value &

                (dataframe['close'] < dataframe['ema_200']) &
                (dataframe['close'] > dataframe['ema_50']) &
                (dataframe['rsi'] > self.sell_rsi_under_6.value) &
                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.sell_condition_7_enable.value &

                (dataframe['rsi_1h'] > self.sell_rsi_1h_7.value) &
                qtpylib.crossed_below(dataframe['ema_12'], dataframe['ema_26']) &
                (dataframe['volume'] > 0)
            )
        )

        conditions.append(
            (
                self.sell_condition_8_enable.value &

                (dataframe['close'] > dataframe['bb_upperband_1h'] * self.sell_bb_relative_8.value) &

                (dataframe['volume'] > 0)
            )
        )

        """
	for i in self.ma_types:
            conditions.append(
                (
                    (dataframe['close'] > dataframe[f'{i}_offset_sell']) &
                    (dataframe['volume'] > 0)
                )
        )
	"""

        if conditions:
            dataframe.loc[
                reduce(lambda x, y: x | y, conditions),
                'sell'
            ] = 1

        return dataframe


# Elliot Wave Oscillator
def EWO(dataframe, sma1_length=5, sma2_length=35):
    df = dataframe.copy()
    sma1 = ta.EMA(df, timeperiod=sma1_length)
    sma2 = ta.EMA(df, timeperiod=sma2_length)
    smadif = (sma1 - sma2) / df['close'] * 100
    return smadif

HRM

Components
Components form the basis of our HRM system. Each component represents compensation and benefits an employee can receive as part of their salary package. This document describes what components are and how they are used throughout the entire system. As you will see components relate closely to policies, the only difference being that components are based on law, sectoral and institutional rules. They are defined by law and therefor informatively enforce those laws.
Policies
Policies is a concept that drives the application. It is a new paradigm focussed on achieving truly dynamic applications. It is different from usual concepts for applications. Although the reference architecture is one that is widely known and is worked on by each and every organization. The basis for the policy system can be found in the security realm. Consider 2 types of applications: Data driven: Data is highest good, we have input and need to convert it to output Process driven: A process defines how an object moves through its different states, the data in that object becomes secondary Here we define a new way to approach a dynamic system.
Client provisioning
Client provisioning is the process of seting up and deploying a container for a new client. In this project we will focus on self-service setup. The idea is that a client can set up new environments and has access to them via single sign on. Process CamundaClient provisioning The process needs to differentiate between a new environment and an existing one. The steps to create or update an environment differ.
Dynamic routes
Dynamic routes are used to load layouts based on a structure in url format. These simplify the work needed to setup a page and allow for easy communication between dynamic components. Requirements Fixed structure Clean URL compliance wiki Unlimited length Component parameter support Pass information up/down the component tree Support commands Fixed structure Consider this url as an example: https://my-tool.org/employees/details/id/aaa-aaa-aaa/account/bbb-bbb-bbb/transaction/ccc-ccc-ccc The url has the following parts: host https://my-tool.org: this denotes the application url.

Components

Components form the basis of our HRM system. Each component represents compensation and benefits an employee can receive as part of their salary package. This document describes what components are and how they are used throughout the entire system. As you will see components relate closely to policies, the only difference being that components are based on law, sectoral and institutional rules. They are defined by law and therefor informatively enforce those laws. Within the system they are immutable, although they can be influenced by policy. Law provides us with constraints where organizations are free to define their own policies within these constraints. The first order of business is providing a system that can capture these components and their constraints. This is the job of components.

Note

The difference between components and policies is:

Components have a legal character, follow institutional law and are therefor immutable. They are controlled on the highest level.
Policies is what clients and users can manage to tailor the application to their needs, including the input and results components use and provide.

Why do we need components

Components in our system have a couple of goals. Most essential is that components form the input for salary composition and form the input for payroll engines. The scope of our software is to provide a tool allowing for flexible compensation and benefits for employees. The goal is to make this manageable for the employer by providing in reporting and cost estimations. Additionally, the intent is to create insight for employees into their salary composition by providing a clear overview of all components that make up their salary.

graph TD;
    A[Labour law] --> X(Component)
    B[Sectoral agreements] --> X
    X --> C[Payroll]
    X --> D[Reporting]
    X --> E[Information]

    C ==> F(Benefit in kind)
    C ==> G(Budget impact)
    D ==> H(Total cost of ownership)
    D ==> I(Cost optimization)
    E ==> J(Simulation)
    E ==> K(Visualization)
    E ==> L(Comprehensible)

    classDef green fill:#9f6,stroke:#333,stroke-width:2px,color:black;
    classDef orange fill:#f96,stroke:#333,stroke-width:4px;
    class X green
    class di orange

Example component

To clarify components, we will take the example of an electric bike. Let’s crunch the numbers. In this example an employee has chosen for an electric bike with a consumer price equal to 2799.00.

	Result	Formula
residualValue	2351.16	consumerPrice - (consumerPrice * residualPercentage)
RSZemployee	307.30	residualValue * RSZemployeePercentage
companyTax	817.55	((residualValue - RSZemployee) * companyTaxPercentage
cityTax	0.00	((residualValue - RSZemployee) * companyTaxPercentage * cityTaxPercentage
Netto	1226,32	residualValue - RSZemployee - companyTax - cityTax

Additionally we want to calculate the budget value.

	Result	Formula
tax	0	consumerPrice * taxRate
residualValue	447.84	(consumerPrice - tax) * residualPercentage
RSZemployerVAA	0	VAAbicyle * componentPeriod * companyContribution
companyTax	817.55	((residualValue - RSZemployee) * companyTaxPercentage
budgetValue	2351.16	(consumerPrice - tax - residualValue + RSZemployerVAA) / installments

To be able to calculate the values we need, we need some information. These system needs to provide these parameters somehow, or we can’t complete our calculations.

Parameter	Value
consumerPrice	2799.00
residualPercentage	0.16
RSZemployeePercentage	0.1307
companyTaxPercentage	0.40
cityTaxPercentage	0
taxRate	0
VAAbicycle	0
componentPeriod	36
companyContribution	0.28
installments	1

Note

A lot of information can be found here. Without specifying where the values come from, we can see that we need about 10 parameters to be able to calculate a single component. Notice that we don’t specify where these parameters are coming from. This is because these parameters differ based on:

sector
product classification
tax rates
contract
residential address
employer contributions
…

This requirement is where policies come in. We need a way to resolve these values given a set of conditions. Additionally, we need to be able to get these parameters from different corners of the application, we need to be able to set default values, we need to be able to overwrite them to allow for cost optimization and much more. The only mechanism than can cover these use cases is policies. Check out the policies page for more information.

Defining formulas and parameters is all well and good. Although without a way to trigger calculations, or to resolve values, the ability to do calculation has no value. We need to solve a few more problems. Firstly we need to define the input and output of a component.

Component IO

To begin to describe how calculations behave within the system, we need a use case. The first use case we will be considering is that of flexible salary. Flexible salary allows employees to assign a part of their income to their own discretion. Al be it, as always, within the boundaries of the law. To achieve this, we need to link components to employees and/or their contracts. As we determined, components make up an employees salary. Therefor it is the declaration of these components in relation with these employees that starts it all. The question becomes, what are we linking, and how do we identify links? To answer that question, we will be linking multiple concepts through the same system, e.g.:

Legal requirements: Monthly/hourly salary
Flexible products chosen/ordered by the employee
Compensation and benefits as negotioated with the employer
…

The thing to notice is that the link can take many forms. Therefor basing it on one of these forms would not be practical. Eventually we would need to implement logic that rhymes these corners of the application. The result would be messy, trying to align different use cases together, in a logical, sequential and potentially impact with legal ramifications, could result in: high cost for analysis, slowing progress and complex unmaintainable code.

To mitigate these issues, we will define a standard input/output format for the formulas. One that allows for linking to any and all other concepts both internal and external to the own organization. This would allow for a highly standardized approach that will help to focus on the value components provides. The responsibility of connecting to the component engine will be relayed to the using side. It would be best placed there since it adheres to encapsulation principles.

So we need to define our own object to use as input and output for the component system.

Input

The input for calculations is the actual link. It will contain all values available from the origin. And it will add the configuration for the engine to process the calculations.

Note

The one constant to all links, is the component. It ties it all together.

Important

To keep documentation consistent and readable for non-technical people, I will be defining the objects as yaml. However the system will use json as a format under the hood.

Let’s define an object that can contain the links we need for the calculation engine. Additionally, input is required for calculations to be possible. Therefor when creating the entry, we will add the values as they are available.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


kind: component-values
component: mobilePhone          # component technical name
version: latest                 # indicates to always use the latest version for calculation
date: 20240101100000            # date the entry was created
origin: flexplan-order          # reference to the origin
origin-id: 1234                 # id for the reference
links:
  - employee: aaa-aaa-aaa       # employee linked to the order
  - flexOrder: xxx-xxx-xxx      # order that created this link
  # more links are possible
values:
  - consumerPrice: 2799.00      # product price as chosen by employee in the order
  - residualPercentage: 0.16    # residual percentage as part of a lease/order
  - taxRate: 0.0                # tax rate provided by the product categorization

At this point an entry is made but no calculations have been performed. We need to define how calculations are handled. It is up to the component to tell the system how the calculations should behave. This depends on the context of the component. Let’s add some example calculation definitions to the entry:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


kind: component-values
component: mobilePhone          # component technical name
version: latest                 # indicates to always use the latest version for calculation
date: 20240101100000            # date the entry was created
origin: flexplan-order          # reference to the origin
origin-id: 1234                 # id for the reference
links:
  - employee: aaa-aaa-aaa       # employee linked to the order
  - flexOrder: xxx-xxx-xxx      # order that created this link
  # more links are possible
input:
  - consumerPrice: 2799.00      # product price as chosen by employee in the order
  - residualPercentage: 0.16    # residual percentage as part of a lease/order
  - taxRate: 0.0                # tax rate provided by the product categorization
config:
  start: 20240101               # date calculations should start
  end: 20250101                 # date calculations should end
  interval: * * 1 * *           # cron expression indicating recurring calculations

Policies

Policies is a concept that drives the application. It is a new paradigm focussed on achieving truly dynamic applications. It is different from usual concepts for applications. Although the reference architecture is one that is widely known and is worked on by each and every organization. The basis for the policy system can be found in the security realm.

Consider 2 types of applications:

Data driven: Data is highest good, we have input and need to convert it to output
Process driven: A process defines how an object moves through its different states, the data in that object becomes secondary

Here we define a new way to approach a dynamic system. Policies are the highest goods and they influence the data/fields shown in the UI. They define the process to follow, security to be applied, where data can be resolved, and much more.

How can we use all fields across our application and have them influence all other aspects of the system? The answer is policies.

Note

Reference architecture can be found in any/all security systems. We will use firewall rules and policies as an example to model this system.

Policy Definition Language (PDL)

Following the example within security, policies are defined in a single file. They have a specific syntax that is flexible and extensible. The base of the system is this ruleset of policies. It will drive the application.

General structure could be:

FOR [TARGET] WHEN [SELECTOR]
    RESULT [TARGET SPECIFIC KEYS]
    ON [DATETIME] TO [DATETIME]
    ROLE [ROLE NAMES] ;

Our system is mostly Json based, so json would be a better fit. It would be easier to integrate with the existing software and methodologies. The downside is that it is not really readable for non-technical people. The idea is that knowledgeable people can quickly perform changes to this file. Therefor Yaml would be a better fit. It allows for easy conversion to and from Json and has the added benefit that it is more readable for non-technical people. So our implementation will be Yaml based.

Important

The policies will have a json structure trough the entire system, except for the policy file editor. There a conversion will be made to Yaml.

The structure from above in Yaml would have this structure:

1
2
3
4
5
6
7


policies:
  - for: <target>
    when: <condition>
    on: <datetime>
    to: <datetime>
    role: <role_list>
    result: <result resolver>

Parts explained:

TARGET: Denotes the effect of a policy (e.g: filtering lists, defining parameters, security constraints, …)
TARGET SPECIFIC KEYS: Each target has its own specific keys/implementation. This is where they are defined.
ON and TO (optional): Allows for versioning policies by date and time
ROLE NAMES (optional): Specifies specific roles for whom the rule applies

Comments

You can add comments to the file with the # character.

1
2
3
4
5
6
7


# This is a comment
policies:
  - for: parameter # This is another comment
    selector: "mobilePhone.benefitInKind"
    result:
      type: static
      value: 3

Targets

Parameter

Parameters are used inside (payroll) components. Components denote different aspects that make up the compensation and benefits an employee receives in exchange for their labour/services. Business requirement states that these parameters can come from anywhere. PDL is dynamic and allows for resolving these parameters from different sources. The job of the PARAMETER query is to describe how to resolve these parameters.

Important

For resolving paranmeters, we need a way to define a value for all components. Therefor the parameter selector will support wildcards. E.g.: when: *.companyVat. This specifies that all parameters named companyVat will have the same value. Notice that we can still overwrite the value if the need should arise.

Static resolver

The static resolver is perhaps the simplest. It defines the value as part of the policy.

e.g.:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


policies:
  - for: parameter
    selector: "mobilePhone.benefitInKind"
    result:
      type: static
      value: 3
  - for: parameter
    selector: "mobileSubscription.benefitInKind"
    result:
      type: static
      value: 3

The above policies define a fixed value. This value will be applicable to all employees since the WHEN specification was omitted. The effect is that in every calculation of the components mobilePhone and mobileSubscription will have a benefitInKind with the value of 3.

Considder the following adaptations:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


policies:
  - for: parameter
    selector: "mobilePhone.benefitInKind"
    when: "employee.function.type" = "management"
    result:
      type: static
      value: 4
    
  - for: parameter
    selector: "mobilePhone.benefitInKind"
    result:
      type: static
      value: 3

To note here is that both rules affect the same parameter. Rules are processed in order as found in the ruleset. The effect off the above configuration is:

if the function of the employee is of type management then the result of the benefitInKind for a mobile phone will be 4
for all other employees the result would be 3

Important

Rules need to be ordered correctly, from most relevent to least relevant.

Considder the following adaptation:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


policies:
  - for: parameter
    selector: "mobilePhone.benefitInKind"
    when: "employee.function.type" = "management"
    on: 20200101000000
    to: 20210101000000
    result:
      type: static
      value: 2
  
  - for: parameter
    selector: "mobilePhone.benefitInKind"
    when: "employee.function.type" = "management"
    result:
      type: static
      value: 4
    
  - for: parameter
    selector: "mobilePhone.benefitInKind"
    result:
      type: static
      value: 3

To note here is the addition of the on and to parameters that limit the policy in time. The effect of these rules are:

Before januari 1st 2020 all employees in management would have a benefit in kind for a mobile phone equal to 4. The first rule does not evaluate to true because of the date specification.
For the entire year 2020 all employees in management would have a benefit in kind for a mobile phone equal to 2. The rule applies since the date falls betwween the dates specified in the ON and the TO date constraints.
All other employees, at any given time will have a benefit in kind of 3

Important

To help with the dates, the dates should be handled as:

on date is inclusive
to date is exclusive

This allows for easy definition of start dates, and eliminates the need to do arithmatic on the end date to subtract one tick.

Entity resolver

The entity resolver will implement a method that allows values to be obtained from other parts of the system. It effectively allows linking a parameter to a field anywhere in the application.

1
2
3
4
5
6
7


policies:
  - for: parameter
    selector: "mobilePhone.benefitInKind"
    result:
      type: entity
      selector: "com.platfohrm.employer.BikMobilePhone"
      where: "com.platfohrm.employer.Id" = "${{env.EmployerId}}"

This policy has the effect of resolving the benefitInKind parameter for the mobilePhone component from the entity com.platfohrm.employer.BikMobilePhone. To achieve this the service responsible for the employer data needs information to retrieve the correct record. Here we specified that the records Id should equal the EmployerId of the environment where this ruleset is active.

Important

The environment variables are specified on the top of the file. You can define variables and use them throughout the ruleset. This allows for copying over rules between environments. You would only need to override the variables for each environment. You can use the following format:

1
2
3


env:
  - EmployerId: 1234
  - EnvironmentId: abcd

Hierarchy

An important reason for this implementation is the concept of hierarchies. To explain hierarchies we first need to look to the structure of clients of the application.

The structure is as follows:

---
title: Hierarchy
---
graph TD;
    A[Platfohrm] -->|has| B(Expert A)
    A -->|has| C(Expert B)
    B -->|supports| E[Employer 1]
    B -->|supports| G[Empoloyer 2]
    C -->|supports| H[Employer 4]
    C -->|supports| I[Employer 5]
    G -->|devided in| J(Establishment A)
    G -->|devided in| K(Establishment B)
    K -->|structures| L(Department 1)
    K -->|structures| M(Department 2)
    L -->|employs| O(Employee A)
    L -->|employs| P(Employee B)
    P -->|contract| Q[Contract 1]
    P -->|contract| R[Contract 2]

Requirement: Policies from the top level are applied, unless they are overwritten on a lower level.

Important

To achieve hierarchy we need a file that structures the policies in sequence. This ties together with rules that are structured from most relevant to least relevant.

Now lets structure the policies as they are defined following this hierarchy.

Remember: The first rule to match will be applied.

We will start to build this hierarchical structure from the ground up. We will start with an example to determine the cost for mealVouchers. In our example this would represent a component that is part of the compensation for an employee.

Level 0: Platfohrm

So lets start from the top down. At the top level we define that the value of one meal voucher is 5.00. The business reason for specifying this can be all sorts. In this case we would assume its the legal minimum. So the policy for Platfohrm would become:

1
2
3
4
5
6


policies:
  - for: parameter
    selector: "mealVoucher.value"
    result:
      type: static
      value: 5.00

Let’s say law changes starting the 1st of 2025. The value would become 6.00 due to regulatory changes. To achieve this, we would change the policy to:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


policies:
  - for: parameter
    selector: "mealVoucher.value"
    to: 20250101
    result:
      type: static
      value: 5.00
      
  - for: parameter
    selector: "mealVoucher.value"
    on: 20250101
    result:
      type: static
      value: 6.00

Important

Note the TO and ON parts of the policies above. When TO is specified without a corresponding ON. It means that all dates before the TO date will evaluate this condition as true. In contrast to this, a ON without a TO denotes that there is no end date specified for the given policy. All dates starting from the given date would evaluate to true.

The effect of this policy is that prior to 20250101 the value would always be 5.00. Starting from 20250101 the value would always be 6.00.

If you want to specify more constraints like the function classification of an employee, you can look at the previous sections of this documentation.

To keep this example simple we will continue to the next level.

Level 1: Expert

The next level in the hierarchy is that of the expert. This organisation can again specify their own policies. For example: the expert wants to create a more attractive payroll packet for staff in an accounting function. One of the measures they take, is to change the value of the meal vouchers. This would result in this policy:

FOR PARAMETER "mealVoucher.value" WHEN "employee.function.type" = "accounting" RESULT STATIC 10.00

1
2
3
4
5
6
7


policies:
  - for: parameter
    selector: "mealVoucher.value"
    when: "employee.function.type" = "accounting"
    result:
      type: static
      value: 10.00

This policy states that when employees occupy a function of type accounting then meal vouchers would get a value of 10.00.

Level 2: Employer

The next level in the hierarchy is that of the employer/client. He is of the opinion that 10.00 is to high. His organization is an accountancy firm and that would cost to much. Therefor the employer wants to change this value to 8.00. To achieve this he creates a policy:

1
2
3
4
5
6
7


policies:
  - for: parameter
    selector: "mealVoucher.value"
    when: "employee.function.type" = "accounting"
    result:
      type: static
      value: 8.00

Note: The policy created here is the exact same as the Expert, but with a different value.

Level 3: Establishment

The organization has many establishments, but one of their establishment has an in house kitchen. They serve hot meals every day at discounted prices. Management feels that this is already a big plus for the employee at this location and wants to change the value of the meal vouchers accordingly. The employer would add a new policy to achieve this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


policies:
  - for: parameter
    selector: "mealVoucher.value"
    when: "employee.function.type" = "accounting" AND "employee.contract.location.city"
    result:
      type: static
      value: 6.00
      
  - for: parameter
    selector: "mealVoucher.value"
    when: "employee.function.type" = "accounting"
    result:
      type: static
      value: 8.00

Important

Note that the new policy was added above the previous policy. It is more specific and therefor must be above the other statements. This because the first to match will be applied.

Level 4: Department

The organization has sales representatives on the road and would like to support the expensive food prices by increasing the value of the meal vouchers. The organization would update the policy:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21


policies:
  - for: parameter
    selector: "mealVoucher.value"
    when: "employee.function.type" = "accounting" AND "employee.contract.location.city"
    result:
      type: static
      value: 6.00
      
  - for: parameter
    selector: "mealVoucher.value"
    when: "employee.function.type" = "accounting"
    result:
      type: static
      value: 8.00

  - for: parameter
    selector: "mealVoucher.value"
    when: "employee.department.name" = "sales"
    result:
      type: static
      value: 9.00

Level 5: Individual employee

The employer got the chance to attract a senior sales representative. This person has a proven track record and getting him to join the company had proven tough. Negotiations were tough. One of the extra benefits this person asked was for a higher value of meal vouchers. He/she negotiated a value of 10. The company updated their policy to reflect this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28


policies:
  - for: parameter
    selector: "mealVoucher.value"
    when: "employee.Id" = "1234"
    result:
      type: static
      value: 10.00
      
  - for: parameter
    selector: "mealVoucher.value"
    when: "employee.function.type" = "accounting" AND "employee.contract.location.city"
    result:
      type: static
      value: 6.00
      
  - for: parameter
    selector: "mealVoucher.value"
    when: "employee.function.type" = "accounting"
    result:
      type: static
      value: 8.00

  - for: parameter
    selector: "mealVoucher.value"
    when: "employee.department.name" = "sales"
    result:
      type: static
      value: 9.00

Important

The individual rule was placed on top to make sure it was applied before other rules were processed.

Level 6: Individual contract

Since the acquisition of the new sales representative sales were going great. So great the company could hardly keep up. This meant some of the sales representatives have been asked to work part-time. Their contract was amended to a part-time function. One of these employees had experience in ICT and agreed to take on additional work as a support agent within the ICT department. HR drew up an additional contract for the employee, again for a part-time, but this time in the ICT department as a support agent. An agreement was made that this person would get a value 7.50 for his meal vouchers while working the function of support agent. The company added a policy to the new contract:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35


policies:
  - for: parameter
    selector: "mealVoucher.value"
    when: "contract.Id" = "abcd"
    result:
      type: static
      value: 7.50
      
  - for: parameter
    selector: "mealVoucher.value"
    when: "employee.Id" = "1234"
    result:
      type: static
      value: 10.00
      
  - for: parameter
    selector: "mealVoucher.value"
    when: "employee.function.type" = "accounting" AND "employee.contract.location.city"
    result:
      type: static
      value: 6.00
      
  - for: parameter
    selector: "mealVoucher.value"
    when: "employee.function.type" = "accounting"
    result:
      type: static
      value: 8.00

  - for: parameter
    selector: "mealVoucher.value"
    when: "employee.department.name" = "sales"
    result:
      type: static
      value: 9.00

This completes the example of hierarchies. All that is left is putting it all together.

Putting it all together

Policy file structure

For our example of meal vouchers. All levels have created their policies. All that is left is combining these policies so that the correct policy would apply for each level of the hierarchy. The answer is simple, we load all policy files in reverse order, from the bottom of the hierarchy to the top. This would result in the following policy:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55


policies:
  # Contract level
  - for: parameter
    selector: "mealVoucher.value"
    when: "contract.Id" = "abcd"
    result:
      type: static
      value: 7.50

  # Employee level
  - for: parameter
    selector: "mealVoucher.value"
    when: "employee.Id" = "1234"
    result:
      type: static
      value: 10.00
      
  # Department level
  - for: parameter
    selector: "mealVoucher.value"
    when: "employee.department.name" = "sales"
    result:
      type: static
      value: 9.00

  # Establishment level
  - for: parameter
    selector: "mealVoucher.value"
    when: "employee.function.type" = "accounting" AND "employee.contract.location.city"
    result:
      type: static
      value: 6.00
      
  # Employer level
  - for: parameter
    selector: "mealVoucher.value"
    when: "employee.function.type" = "accounting"
    result:
      type: static
      value: 8.00

  # Expert level
  - for: parameter
    selector: "mealVoucher.value"
    when: "employee.function.type" = "accounting"
    result:
      type: static
      value: 10.00
      
  # Platfohrm level
  - for: parameter
    selector: "mealVoucher.value"
    result:
      type: static
      value: 5.00

This structure will ensure that the policies are applied for the correct conditions.

Note

The top level platfohrm can define default values for almost any parameter in the system. As long as there is no policy that overrides this, the top level would always be applied.

Adapting over time

One last part of the system has to do with changes in policy on any of the levels. To achieve this we can use a similar approach as we did for the environment variables. We would start the policy file by specifying the variables on top and add a similar structure for the hierarchy:

1
2
3
4
5
6


hierarchy:
  - base: "platfohrm:latest"
  - expert: "expertA:latest"
env:
  - EmployerId: 1234
  - EnvironmentId: abcd

The above file specifies that additional policy sets are loaded from other levels. This employer works with the base as defined by Platfohrm. And they have chosen for ExpertA. The implementation will process this HIERARCHY block and add the files as specified above.

Note

Note the :latest addition. This specifies that the latest version of these files will be included. If needed, an organization can choose to pin it to a previous version. The entry can then be changed to BASE "platfohrm:3.0.2 and that ruleset will always be applied. Updates would not be automatic until the client updates the version, or specifies latest again.

The entire file would look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50


kind: policy-definition
version: 1.0
name: clientA default policy
description: You can describe the policy here
hierarchy:
  - base: "platfohrm:latest"
  - expert: "expertA:latest"
env:
  - EmployerId: 1234
  - EnvironmentId: abcd
policies:
  # Contract level
  - for: parameter
    selector: "mealVoucher.value"
    when: "contract.Id" = "abcd"
    result:
      type: static
      value: 7.50

  # Employee level
  - for: parameter
    selector: "mealVoucher.value"
    when: "employee.Id" = "1234"
    result:
      type: static
      value: 10.00

  # Department level
  - for: parameter
    selector: "mealVoucher.value"
    when: "employee.department.name" = "sales"
    result:
      type: static
      value: 9.00

  # Establishment level
  - for: parameter
    selector: "mealVoucher.value"
    when: "employee.function.type" = "accounting" AND "employee.contract.location.city"
    result:
      type: static
      value: 6.00

  # Employer level
  - for: parameter
    selector: "mealVoucher.value"
    when: "employee.function.type" = "accounting"
    result:
      type: static
      value: 8.00

Important

Notice that the policies that are included in via the hierarchy are not present in this file. They will be added automatically. Here we focus on 1 file for a particular client.

User Interface

Dynamic UI

1
2
3
4
5
6
7
8


policies:
  - for: parameter
    selector: "mobilePhone.benefitInKind"
    when: "contract.Id" = "abcd"
    result:
      type: entity
      selector: "com.platfohrm.employer.BikMobilePhone"
      where: "com.platfohrm.employer.Id" = "${{env.EmployerId}}"

Our example above defines that the entity com.platfohrm.employer should have a field BikMobilePhone. The type of the field can be deduced from the parameter mobilePhone.benefitInKind. This definition allows the system to dynamically add this field to the UI responsible for the employer data. All that was needed is this one line policy definition.

Working with policies

In a first stage we will start from the file itself. Although in a later phase we will be able to define a UI interface to work with the policies. The input of a policy UI is always this file. It can be filtered to show the relevant policies in accordance with the subject on the screen. Let’s consider a UI for definition of policies related to components.

Given this ruleset:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


policies:
  - for: parameter
    selector: "mobilePhone.benefitInKind"
    when: "contract.Id" = "abcd"
    result:
      type: static
      value: 3
        
  - for: parameter
    selector: "mobileSubscription.benefitInKind"
    when: "contract.Id" = "abcd"
    result:
      type: static
      value: 3

When the user has the component mobilePhone open, then the filter for the policies is:

All policies of type PARAMETER where name starts with mobilePhone.
(optional) filter by date range:
- Active today: all that have no ON section or today’s date is between ON and TO of the date section

Versioning

Minimal policy

For a client to start using the system, the minimum requirement is that a base is defined. The example bellow denotes the absolute minimum for the system to operate. In this example the client follows the latest version of the platfohrm base.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


kind: policy-definition
version: 1.0
name: clientA default policy
description: You can describe the policy here
hierarchy:
  - base: "platfohrm:latest"
env:
  - EmployerId: 1234
  - EnvironmentId: abcd
policies: []

Splitting policy sets

Policy sets can become large, additionally we might want to define different defaults depending on a clients sector or wishes. We can achieve this by creating multiple files and combining them. Let’s apply the same principles for the most important levels and set this up.

Defining a base

Let’s say we want to provide a different base for construction oriented organizations. We will first create our base, one that we will use for all non-construction oriented organizations.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


kind: policy-definition
version: 1.0
name: platfohrm-base
description: We recommend this policy for all non-construction oriented organizations.
hierarchy: [] # Hierarchy is empty because we are defining a base
env: [] # No environment variables needed at this point
policies:
  - for: parameter
    selector: "mealVoucher.value"  # As an example we will provide a value for meal vouchers
    result:
      type: static
      value: 5.00
  - for: parameter
    selector: "mobilePhone.benefitInKind"  # We have other policies defined
    result:
      type: static
      value: 3
  - for: parameter
    selector: "mobileSubscription.benefitInKind"  # We have other policies defined
    result:
      type: static
      value: 3

Now let’s define an entirely different base for construction oriented organizations.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


kind: policy-definition
version: 1.0
name: platfohrm-base-construction
description: Base tailored to the construction sector
hierarchy: [] # Hierarchy is empty because we are defining a base
env: [] # No environment variables needed at this point
policies:
  - for: parameter
    selector: "mealVoucher.value"  # As an example we will provide a value for meal vouchers
    result:
      type: static
      value: 8.00   # We change the value to 8
  - for: parameter
    selector: "mobilePhone.benefitInKind"  # We have other policies defined
    result:
      type: static
      value: 3
  - for: parameter
    selector: "mobileSubscription.benefitInKind"  # We have other policies defined
    result:
      type: static
      value: 3

Caution

Here we changed the entire ruleset to tailor for construction oriented organizations. We had to define unchanged policies as well. The problem with this is manageability. If we have to create a base for all sectors, we would have a lot of maintenance propagating our changes through all these different bases.

A better way to achieve the same result but in a more manageable manner would be to extend our own default base. Doing so we would only need to define the changes for the construction instead of duplicating all policies. The result would be:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


kind: policy-definition
version: 1.0
name: platfohrm-base-construction
description: Base tailored to the construction sector
hierarchy: 
  - base: platfohrm-base:latest
env: [] # No environment variables needed at this point
policies:
  - for: parameter
    selector: "mealVoucher.value"  # As an example we will provide a value for meal vouchers
    result:
      type: static
      value: 8.00   # We change the value to 8

Using this approach we would get all changes from the base. We just needed to define the changes.

Note

Note that we chose to follow the latest version of our platfohrm-base. The effect is that new versions would automatically impact our version for the construction sector. If we want to avoid that, all we need to do is change platfohrm-base:latest to platfohrm-base:1.0. That would give us even more control.

A client can now choose what base to follow. The organization can define their base in their own policy definition:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


kind: policy-definition
version: 1.0
name: My organization
description: You can describe the policy here
hierarchy:
  - base: platfohrm-base-construction:latest
env:
  - EmployerId: 1234
  - EnvironmentId: abcd
policies: []

In this example, the organization decided to follow the platfohrm-base-construction and follow all changes automatically when new versions of the base are released.

Cherry-picking

Cherry-picking is the process where we can really fine-tune the needs of the policies. As an example, a car policy might be a difficult one. We as an organization want to offer more choices to the organizations using our software. Let’s say we want to define 2 car policies.

Generous car policy:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


kind: policy-definition
version: 1.0
name: platfohrm-car-policy-generous
description: Car policy allowing for a higher catalog price
hierarchy: [] # Hierarchy is empty because we are defining a base
env: [] # No environment variables needed at this point
policies:
  - for: parameter
    selector: "carPolicy.catalogPrice"  # As an example we will set a maximum for the catalog price
    result:
      type: static
      max: 36000   # We change the value to 36k

Here we defined the generous car policy, we have set the maximum catalog price for a car to be 36 000.

Budget car policy:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


kind: policy-definition
version: 1.0
name: platfohrm-car-policy-budget
description: Budget friendly car policy
hierarchy: [] # Hierarchy is empty because we are defining a base
env: [] # No environment variables needed at this point
policies:
  - for: parameter
    selector: "carPolicy.catalogPrice"  # As an example we will set a maximum for the catalog price
    result:
      type: static
      max: 28000   # We change the value to 32k

Here we have defined the budget car policy. We lowered the maximum catalog price to 28 000.

We can now define our base in similar fashion. Since this base will be widely used, we will add the budget car policy to the base.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


kind: policy-definition
version: 1.0
name: platfohrm-base
description: We recommend this policy for all non-construction oriented organizations.
hierarchy:
  - expert: platfohrm-car-policy-budget:latest
env: [] # No environment variables needed at this point
policies:
  - for: parameter
    selector: "mealVoucher.value"  # As an example we will provide a value for meal vouchers
    result:
      type: static
      value: 8.00   # We change the value to 8
  - for: parameter
    selector: "mobilePhone.benefitInKind"  # We have other policies defined
    result:
      type: static
      value: 3
  - for: parameter
    selector: "mobileSubscription.benefitInKind"  # We have other policies defined
    result:
      type: static
      value: 3

Here we used our previous example of our base. We have now added the budget car policy to it. With cherry-picking we can go even further. Let’s create 2 more policies. One for meal vouchers and one for mobile phone.

Meal vouchers:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


kind: policy-definition
version: 1.0
name: platfohrm-meal-vouchers
description: Legal requirements when working with meal vouchers
hierarchy: []
env: [] # No environment variables needed at this point
policies:
  - for: parameter
    selector: "mealVoucher.value"  # As an example we will provide a value for meal vouchers
    result:
      type: static
      min: 4.50   # We set the minimum to 4.5
      max: 10.00  # We set the maximum to 10

We have set constraints on the meal vouchers. The policy above states that the value of meal vouchers needs to be between a min and a max: 4.50 and 10.00 respectively.

Mobile phone:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


kind: policy-definition
version: 1.0
name: platfohrm-mobile-phone
description: Mobile phone policy defaults
hierarchy: []
env: [] # No environment variables needed at this point
policies:
  - for: parameter
    selector: "mobilePhone.benefitInKind" 
    result:
      type: static
      value: 3
  - for: parameter
    selector: "mobileSubscription.benefitInKind" 
    result:
      type: static
      value: 3

Here we specify that the benefit in kind for a mobile phone always equals 3.00, and that the benefit in kind for a mobile subscriptionalso always equals3.00```.

Combining all of the above, we can now specify our base policy in a very clean and maintainable way:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


kind: policy-definition
version: 1.0
name: platfohrm-base
description: We recommend this policy for all non-construction oriented organizations.
hierarchy:
  - expert: platfohrm-car-policy-budget:latest
  - expert: platfohrm-mobile-phone:latest
  - expert: platfohrm-meal-vouchers:latest
env: [] # No environment variables needed at this point
policies: []

Here we have defined a base for organizations and experts to build upon. We cherry-picked policies and combined them to an all-inclusive policy base. An organization or an expert can now build upon this policy freely and add value to the policy as is needed. As a final example let’s say an expert might want to change the base policy and upgrade the car policy to the generous car policy. All he would have to do is define this in a policy file. The expert can define their own policies easily. AS an example let’s say the expert defines a policy for accounting oriented organizations.

1
2
3
4
5
6
7
8
9


kind: policy-definition
version: 1.0
name: expertA-accounting-policy-base
description: We recommend this policy for all non-construction oriented organizations.
hierarchy:
  - base: platfohrm-base:latest
  - expert: platfohrm-car-policy-generous:1.0
env: [] # No environment variables needed at this point
policies: []

Here the expert created his own base. He started from the platfohrm-base but changed the car policy to the generous car policy. He also fixed the version of the car policy to 1.0 because he wanted more control over potential changes to provide even better support towards his clients. When he creates new clients in the system. The expert can now set this policy as a base for the client:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


kind: policy-definition
version: 1.0
name: My organization
description: You can describe the policy here
hierarchy:
  - base: expertA-accounting-policy-base:latest
env:
  - EmployerId: 1234
  - EnvironmentId: abcd
policies: []

The client will now follow the policy of expert A. Additional policies can be defined if needed, the client can start to tailor the policies to his needs. This completes the circle.

Important

Here the expert policy is set up as a base meaning that it inherits all defaults of the platfohrm-base. If this was not the case, the system would not work. But the expert is in control when creating the new client. He could have simple added the platfohrm-base to the policy definition of his client and achieve the same result.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


kind: policy-definition
version: 1.0
name: My organization
description: You can describe the policy here
hierarchy:
  - base: platfohrm-base:latest
  - base: expertA-accounting-policy-base:latest
env:
  - EmployerId: 1234
  - EnvironmentId: abcd
policies: []

Note

An expert is in control of his clients. And he can set up clients by defining policies. These policies can be tailored to the clients needs. With this approach we can allow experts to set up clients through a wizard, or allow experts to change configuration for clients. All fine-tuning is done through policies.

Client provisioning

Client provisioning is the process of seting up and deploying a container for a new client. In this project we will focus on self-service setup. The idea is that a client can set up new environments and has access to them via single sign on.

Process

CamundaClient provisioning

The process needs to differentiate between a new environment and an existing one. The steps to create or update an environment differ.

To create a new environment:

Deploy environment using ArgoCD and Kustomize
Create realm in Keycloak
Create Keycloak client for the frontend
Create identity providers
Configure the security headers
Create the admin user
Send password update mail to admin user

When updating the environment:

Update the SMTP server
Configure Keycloak themes
Update locales
Update API cliens and roles

Dynamic routes

Dynamic routes are used to load layouts based on a structure in url format. These simplify the work needed to setup a page and allow for easy communication between dynamic components.

Requirements

Fixed structure
Clean URL compliance wiki
Unlimited length
Component parameter support
Pass information up/down the component tree
Support commands

Fixed structure

Consider this url as an example:

https://my-tool.org/employees/details/id/aaa-aaa-aaa/account/bbb-bbb-bbb/transaction/ccc-ccc-ccc

The url has the following parts:

host https://my-tool.org: this denotes the application url. It is the entry point into the application
layoutSection employees: used to group together layouts, and provides more context from a URL to the user
layoutName details: specifies that the details layout from the group employees should be loaded
data id/aaa-aaa-aaa/account/bbb-bbb-bbb/transaction/ccc-ccc-ccc: The rest of the url is data that has no effect on the layout. This data will be passed down to the components.

This gives us the following structure of URLs:

<host>/<layoutSection>/<layoutName>[/<key>/<value>]*

The key/value pairs are not required and the url can supply as many as is needed.

We will omit the host part for the remainder of this document

Implementation

Loading the layout

In the frontend a page will be created that captures the entire url. The page will load a dynamic layout from the backend given the layoutSection and layoutName.

Important

Considder this example: /employees. This url does not contain a layoutName section and loading would fail. A decision can be made to default the layout name to dashboard when none is provided. Only if this layout is not available from the backend will we redirect to the 404 page.

Passing data

The dynamic layout will then process the remainder of the URL. It will create a key/value list of the pairs in the url and pass it down to the layout itself.

Considder our previous example: employees/details/id/aaa-aaa-aaa/account/bbb-bbb-bbb/transaction/ccc-ccc-ccc

This would load the layout details from employees section. It would then create this key/value list and pass it down:

1
2
3
4
5
6
7


{
  "layoutSection": "employees",
  "layoutName": "details",
  "id": "aaa-aaa-aaa",
  "account": "bbb-bbb-bbb",
  "transaction": "ccc-ccc-ccc"
}

Components from the dynamic layout can use this data as they please. The information would be available when needed.

Important

We include the layoutSection and layoutName in the data. This will be needed to load the correct service to handle requests and commands.

Flexibility

Using URLs this way make them very flexible. The following urls are all equal in how the UI reacts:

employees/details/id/aaa-aaa-aaa/account/bbb-bbb-bbb/transaction/ccc-ccc-ccc
employees/details/account/bbb-bbb-bbb/transaction/ccc-ccc-ccc/id/aaa-aaa-aaa
employees/details/account/bbb-bbb-bbb/id/aaa-aaa-aaa/transaction/ccc-ccc-ccc

As long as the key value pairs stay together the part after the layoutName will load the page correctly.

Command pattern

The dynamic URLs solve a lot of problems by allowing a command pattern through. Considder this url:

employees/details/id/aaaa/do/delete

The component that is responsible for loading id aaaa from the employees service receives an extra command parameter do. this signals the component to perform a delete action for the given id.

At this point the record might be deleted, but the question becomes where do we go next? The layout can’t load this because the record with the given id no longer exists. We have a couple of ways to tackle this:

Default: decide on a default rule to load a layoutName list
Key/value: add data to the url and pass the desired destination

Note

A combination of these 2 will provide the most flexible implementation. Where the system redirects might be different depending on business logic. Therefor it is best to first check for additional data. If that is not present, redirect to the default layout. Redirecting would result in a 404 if the route can not be found and this is the desired behaviour.

To redirect you can add parameters to the url/data:

target-section: the section to redirect to after completion
target-layout: the layout to redirect to for the given section

Note

if the target-section key is not defined, you can redirect to the original section.

Processing actions

To process an action, the first component that understands the action that was requested would execute it. The following steps would occur:

Check if Do parameter exists
1. Check security with the csrf token
2. Execute the action
3. Redirect either to the default layoutName or to the endpoint specified by the data
Load the layout
1. Parse the url
2. Throw error or skip pairs on keys starting with _
3. Load the requested layout

Security

The command pattern allows users to delete records through the url. Therefor we will implement a request forgery implementation. This will make sure that actions can only be triggered from within the UI.

To achieve this action buttons need to do 2 things:

Add a new GUID to the data object
Add the GUID to the action URL

For the component that executes the command the following actions are needed:

Check for the token in the data object.
If it exists, compare it with the token in the url
If it is equal, do the requested action

Upon failure, an error is shown to the user, and the layout does not refresh.

So to implement, the action button adds an _csrfToken to the data:

1
2
3
4
5
6
7
8


{
  "layoutSection": "employees",
  "layoutName": "details",
  "id": "aaa-aaa-aaa",
  "account": "bbb-bbb-bbb",
  "transaction": "ccc-ccc-ccc",
  "_csrfToken": "xxx-xxx-xxx"
}

Then the button can call the url employees/details/id/aaaa/do/delete/csrf/xxx-xxx-xxx. Only when the parameter csrf matches the _csrfToken in the data, will the action be executed.

Important

As a rule, the parameters parsed from the url can never start with a _! This would allow to bypass the security measures. Therefor the mechanism should throw an error when an underscore is found in one of the url parameter keys.

MicroService World

Subsections of MicroService World

Architecture

Subsections of Architecture

Management

Business

Subsections of Business

Git Flow

Application

Technology

Technology

Subsections of Technology

Databases

Subsections of Databases

Elastic search

Index Versioning

Overview

Backup the current index

Create an alias

Kubernetes

Release management

Environments

Production environment

Test/staging/acceptance environment

Development environment

Standards

Semantic Versioning

Git-flow

Git-ops

Conventional commits

Implications

Project management

Open Source

Tools

Subsections of Tools

Github

Github Actions

NuGet Registry

Explaining the code:

Docker container registry

Explaining the code

Jobs

Gita

Gita: a command-line tool to manage multiple git repos

Installation

Auto-completion

Bash

Zsh

Fish

Superman mode

Shell mode

Customization

define repo group and context

add user-defined sub-command using json file

customize the local/remote relationship coloring displayed by the gita ll command

customize information displayed by the gita ll command

customize git command flags

Requirements

Tips

Contributing

Other multi-repo tools

Projects

Subsections of Projects

Trading

Subsections of Trading

Trading Bot

Running the software

Dockerfile

Docker compose file

Config file

Running commands

Backtesting

Hyperopt

Download data

Strategies

NASOSv4

NFI5MOHO_WIP

HRM

Subsections of HRM

Components

customize the local/remote relationship coloring displayed by the `gita ll` command

customize information displayed by the `gita ll` command