feat(fabulinus) add fabulinus (#16909)
**Description** Fabulinus is the server component of the TitanML Takeoff server. ⚒️ Fixes # <!--(issue)--> **⚙️ Type of change** - [X] ⚙️ Feature/App addition - [ ] 🪛 Bugfix - [ ] ⚠️ Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] 🔃 Refactor of current code **🧪 How Has This Been Tested?** <!-- Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration --> **📃 Notes:** <!-- Please enter any other relevant information here --> **✔️ Checklist:** - [X] ⚖️ My code follows the style guidelines of this project - [X] 👀 I have performed a self-review of my own code - [ ] #️⃣ I have commented my code, particularly in hard-to-understand areas - [ ] 📄 I have made corresponding changes to the documentation - [ ] ⚠️ My changes generate no new warnings - [ ] 🧪 I have added tests to this description that prove my fix is effective or that my feature works - [ ] ⬆️ I increased versions for any altered app according to semantic versioning **➕ App addition** If this PR is an app addition please make sure you have done the following. - [ ] 🪞 I have opened a PR on [truecharts/containers](https://github.com/truecharts/containers) adding the container to TrueCharts mirror repo. - [X] 🖼️ I have added an icon in the Chart's root directory called `icon.png` --- _Please don't blindly check all the boxes. Read them and only check those that apply. Those checkboxes are there for the reviewer to see what is this all about and the status of this PR with a quick glance._
This commit is contained in:
Xstar97TheNoob
GitHub
parent
2803de1406
commit
478fb59e26
|
|
@ -0,0 +1,30 @@
|
|||
# Patterns to ignore when building packages.
|
||||
# This supports shell glob matching, relative path matching, and
|
||||
# negation (prefixed with !). Only one pattern per line.
|
||||
.DS_Store
|
||||
# Common VCS dirs
|
||||
.git/
|
||||
.gitignore
|
||||
.bzr/
|
||||
.bzrignore
|
||||
.hg/
|
||||
.hgignore
|
||||
.svn/
|
||||
# Common backup files
|
||||
*.swp
|
||||
*.bak
|
||||
*.tmp
|
||||
*~
|
||||
# Various IDEs
|
||||
.project
|
||||
.idea/
|
||||
*.tmproj
|
||||
.vscode/
|
||||
# OWNERS file for Kubernetes
|
||||
OWNERS
|
||||
# helm-docs templates
|
||||
*.gotmpl
|
||||
# docs folder
|
||||
/docs
|
||||
# icon
|
||||
icon.png
|
||||
File diff suppressed because it is too large
Load Diff
|
|
@ -0,0 +1,37 @@
|
|||
kubeVersion: ">=1.24.0-0"
|
||||
apiVersion: v2
|
||||
name: fabulinus
|
||||
version: 0.0.1
|
||||
appVersion: latest
|
||||
description: Fabulinus is the server component of the TitanML Takeoff server.
|
||||
home: https://truecharts.org/charts/dev/fabulinus
|
||||
icon: https://truecharts.org/img/hotlink-ok/chart-icons/fabulinus.png
|
||||
deprecated: false
|
||||
sources:
|
||||
- https://github.com/truecharts/charts/tree/master/charts/dev/fabulinus
|
||||
- https://hub.docker.com/r/tytn/fabulinus
|
||||
- https://github.com/titanml/takeoff-communitys
|
||||
maintainers:
|
||||
- name: TrueCharts
|
||||
email: info@truecharts.org
|
||||
url: https://truecharts.org
|
||||
keywords:
|
||||
- fabulinus
|
||||
- machine-learning
|
||||
dependencies:
|
||||
- name: common
|
||||
version: 17.2.21
|
||||
repository: oci://tccr.io/truecharts
|
||||
condition: ""
|
||||
alias: ""
|
||||
tags: []
|
||||
import-values: []
|
||||
annotations:
|
||||
max_scale_version: 23.10.2
|
||||
min_scale_version: 23.10.0
|
||||
truecharts.org/SCALE-support: "true"
|
||||
truecharts.org/category: ai
|
||||
truecharts.org/max_helm_version: "3.13"
|
||||
truecharts.org/min_helm_version: "3.12"
|
||||
truecharts.org/train: dev
|
||||
type: application
|
||||
|
|
@ -0,0 +1,28 @@
|
|||
---
|
||||
title: README
|
||||
---
|
||||
|
||||
## General Info
|
||||
|
||||
TrueCharts can be installed as both _normal_ Helm Charts or as Apps on TrueNAS SCALE.
|
||||
However only installations using the TrueNAS SCALE Apps system are supported.
|
||||
|
||||
For more information about this App, please check the docs on the TrueCharts [website](https://truecharts.org/charts/stable/jackett)
|
||||
|
||||
**This chart is not maintained by the upstream project and any issues with the chart should be raised [here](https://github.com/truecharts/charts/issues/new/choose)**
|
||||
|
||||
## Support
|
||||
|
||||
- Please check our [quick-start guides for TrueNAS SCALE](https://truecharts.org/manual/SCALE/guides/scale-intro).
|
||||
- See the [Website](https://truecharts.org)
|
||||
- Check our [Discord](https://discord.gg/tVsPTHWTtr)
|
||||
- Open a [issue](https://github.com/truecharts/charts/issues/new/choose)
|
||||
|
||||
---
|
||||
|
||||
## Sponsor TrueCharts
|
||||
|
||||
TrueCharts can only exist due to the incredible effort of our staff.
|
||||
Please consider making a [donation](https://truecharts.org/sponsor) or contributing back to the project any way you can!
|
||||
|
||||
_All Rights Reserved - The TrueCharts Project_
|
||||
|
|
@ -0,0 +1 @@
|
|||
imageSelector: image
|
||||
|
|
@ -0,0 +1 @@
|
|||
imageSelector: gpuImage
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
---
|
||||
title: Fabulinus Installation Notes
|
||||
---
|
||||
|
||||
Set the `Device` to **CPU** or **GPU** depending on the image selected.
|
||||
|
||||
Set the `Model` to a supported model from the following [list](https://docs.titanml.co/docs/titan-takeoff/experimentation/supported-models).
|
||||
|
||||
Set the `Quant Type` to the correct option when using a particular model.
|
||||
|
||||
Set the `Max Batch Size` to the maximum batch size a model can use.
|
||||
|
||||
::: note
|
||||
|
||||
If you are not using the batching endpoint, /generate, it is best practice to set `Disable Batching` to true. The batching service can interfere with the streaming service, and make it slower.
|
||||
|
||||
:::
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 474 KiB |
|
|
@ -0,0 +1,160 @@
|
|||
# Include{groups}
|
||||
portals:
|
||||
open:
|
||||
# Include{portalLink}
|
||||
path: "/docs"
|
||||
chat:
|
||||
# Include{portalLink}
|
||||
path: "/demos/chat"
|
||||
playground:
|
||||
# Include{portalLink}
|
||||
path: "/demos/playground"
|
||||
questions:
|
||||
# Include{global}
|
||||
# Include{workload}
|
||||
# Include{workloadDeployment}
|
||||
# Include{replicas1}
|
||||
# Include{podSpec}
|
||||
# Include{containerMain}
|
||||
|
||||
- variable: imageSelector
|
||||
label: Select Image
|
||||
schema:
|
||||
type: string
|
||||
default: image
|
||||
enum:
|
||||
- value: image
|
||||
description: CPU
|
||||
- value: gpuImage
|
||||
description: GPU
|
||||
|
||||
# Include{containerBasic}
|
||||
# Include{containerAdvanced}
|
||||
|
||||
- variable: fabulinus
|
||||
group: App Configuration
|
||||
label: Fabulinus Configuration
|
||||
schema:
|
||||
additional_attrs: true
|
||||
type: dict
|
||||
attrs:
|
||||
- variable: device
|
||||
label: Device
|
||||
description: The device to use.
|
||||
schema:
|
||||
type: string
|
||||
default: cpu
|
||||
enum:
|
||||
- value: cpu
|
||||
description: CPU
|
||||
- value: gpu
|
||||
description: GPU
|
||||
- variable: model
|
||||
label: Model
|
||||
description: The model name to use.
|
||||
schema:
|
||||
type: string
|
||||
default: "google/flan-t5-small"
|
||||
required: true
|
||||
- variable: quant_type
|
||||
label: Quant Type
|
||||
description: The quantization type to use.
|
||||
schema:
|
||||
type: string
|
||||
default: int8
|
||||
enum:
|
||||
- value: int8
|
||||
description: Int8
|
||||
- value: float16
|
||||
description: Float16
|
||||
- value: bfloat16
|
||||
description: Bfloat16
|
||||
- value: int8_float16
|
||||
description: Int8_float16
|
||||
- value: int8_bfloat16
|
||||
description: Int8_bfloat16
|
||||
- variable: max_batch_size
|
||||
label: Max Batch Size
|
||||
description: The maximum batch size the model can use.
|
||||
schema:
|
||||
type: int
|
||||
default: 32
|
||||
min: 1
|
||||
- variable: disable_batching
|
||||
label: Disable Batching
|
||||
description: Whether to allow batching or not. Set to False if not using the /generate end point.
|
||||
schema:
|
||||
type: boolean
|
||||
default: true
|
||||
|
||||
# Include{containerConfig}
|
||||
# Include{podOptions}
|
||||
# Include{serviceRoot}
|
||||
# Include{serviceMain}
|
||||
# Include{serviceSelectorLoadBalancer}
|
||||
# Include{serviceSelectorExtras}
|
||||
- variable: main
|
||||
label: "Main Service Port Configuration"
|
||||
schema:
|
||||
additional_attrs: true
|
||||
type: dict
|
||||
attrs:
|
||||
- variable: port
|
||||
label: "Port"
|
||||
description: "This port exposes the container port on the service"
|
||||
schema:
|
||||
type: int
|
||||
default: 10687
|
||||
required: true
|
||||
# Include{serviceExpertRoot}
|
||||
# Include{serviceExpert}
|
||||
# Include{serviceList}
|
||||
# Include{persistenceRoot}
|
||||
- variable: models
|
||||
label: "App Models Storage"
|
||||
description: "Stores the Application Models."
|
||||
schema:
|
||||
additional_attrs: true
|
||||
type: dict
|
||||
attrs:
|
||||
# Include{persistenceBasic}
|
||||
# Include{persistenceList}
|
||||
# Include{ingressRoot}
|
||||
- variable: main
|
||||
label: "Main Ingress"
|
||||
schema:
|
||||
additional_attrs: true
|
||||
type: dict
|
||||
attrs:
|
||||
# Include{ingressDefault}
|
||||
# Include{ingressAdvanced}
|
||||
# Include{ingressList}
|
||||
# Include{securityContextRoot}
|
||||
- variable: runAsUser
|
||||
label: "runAsUser"
|
||||
description: "The UserID of the user running the application"
|
||||
schema:
|
||||
type: int
|
||||
default: 0
|
||||
- variable: runAsGroup
|
||||
label: "runAsGroup"
|
||||
description: "The groupID of the user running the application"
|
||||
schema:
|
||||
type: int
|
||||
default: 0
|
||||
# Include{securityContextContainer}
|
||||
# Include{securityContextAdvanced}
|
||||
# Include{securityContextPod}
|
||||
- variable: fsGroup
|
||||
label: "fsGroup"
|
||||
description: "The group that should own ALL storage."
|
||||
schema:
|
||||
type: int
|
||||
default: 568
|
||||
# Include{resources}
|
||||
# Include{advanced}
|
||||
# Include{addons}
|
||||
# Include{codeserver}
|
||||
# Include{netshoot}
|
||||
# Include{vpn}
|
||||
# Include{documentation}
|
||||
|
|
@ -0,0 +1 @@
|
|||
{{- include "tc.v1.common.lib.chart.notes" $ -}}
|
||||
|
|
@ -0,0 +1 @@
|
|||
{{ include "tc.v1.common.loader.all" . }}
|
||||
|
|
@ -0,0 +1,65 @@
|
|||
image:
|
||||
repository: tytn/fabulinus
|
||||
pullPolicy: IfNotPresent
|
||||
tag: latest-cpu@sha256:f8b30eaa5b61d6085fc715f40619733846927c2266a3e20523e93fb58afeef38
|
||||
gpuImage:
|
||||
repository: tytn/fabulinus
|
||||
pullPolicy: IfNotPresent
|
||||
tag: latest-gpu@sha256:9a826a578ca4157fbe3c091eb78aa35dfca6675f9fae24dcb90495ba4d9715d1
|
||||
|
||||
securityContext:
|
||||
container:
|
||||
readOnlyRootFilesystem: false
|
||||
runAsUser: 0
|
||||
runAsGroup: 0
|
||||
|
||||
service:
|
||||
main:
|
||||
ports:
|
||||
main:
|
||||
protocol: http
|
||||
targetPort: 80
|
||||
port: 10687
|
||||
|
||||
fabulinus:
|
||||
# cpu | gpu
|
||||
device: cpu
|
||||
model: "google/flan-t5-small"
|
||||
# int8 | float16 | bfloat16 | int8_float16 | int8_bfloat16
|
||||
quant_type: int8
|
||||
max_batch_size: 32
|
||||
disable_batching: true
|
||||
|
||||
workload:
|
||||
main:
|
||||
podSpec:
|
||||
containers:
|
||||
main:
|
||||
imageSelector: image
|
||||
probes:
|
||||
liveness:
|
||||
enabled: true
|
||||
type: http
|
||||
path: /docs
|
||||
readiness:
|
||||
enabled: true
|
||||
type: http
|
||||
path: /docs
|
||||
startup:
|
||||
enabled: true
|
||||
type: tcp
|
||||
env:
|
||||
TAKEOFF_DEVICE: "{{ .Values.fabulinus.device }}"
|
||||
TAKEOFF_MODEL_NAME: "{{ .Values.fabulinus.model }}"
|
||||
TAKEOFF_QUANT_TYPE: "{{ .Values.fabulinus.quant_type }}"
|
||||
TAKEOFF_MAX_BATCH_SIZE: "{{ .Values.fabulinus.max_batch_size }}"
|
||||
TAKEOFF_DISABLE_BATCHING: "{{ .Values.fabulinus.disable_batching }}"
|
||||
|
||||
persistence:
|
||||
models:
|
||||
enabled: true
|
||||
mountPath: "/code/models"
|
||||
|
||||
portal:
|
||||
open:
|
||||
enabled: true
|
||||
Loading…
Reference in New Issue