Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to task processing #97

Merged
merged 57 commits into from
Jul 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
311b582
start adding support for task processing API
julien-nc Apr 28, 2024
274b1c8
fix overflow, show message on task error
julien-nc Jun 21, 2024
28e4375
fix failure notification
julien-nc Jun 21, 2024
9d95b3e
allow to cancel a task in the modal while waiting for results
julien-nc Jun 21, 2024
b93ea27
improve style in standalone page
julien-nc Jun 21, 2024
358200a
migrate chatty UI to the task processing API
julien-nc Jun 23, 2024
b838504
fix loading state after canceling a task
julien-nc Jun 24, 2024
d638d3a
fixes #89 sort task types
julien-nc Jul 1, 2024
8f283b0
small style improvements
julien-nc Jul 1, 2024
0af253e
cleanup and start fixing psalm issues
julien-nc Jul 2, 2024
2732ee1
do not show chattyUI tasks in text2text history
julien-nc Jul 3, 2024
2db2455
cleanup input form from old specific contextwrite stuff
julien-nc Jul 3, 2024
b3b66e5
change style of output text field
julien-nc Jul 3, 2024
f11ba49
fixed footer position, expand input/output form and history vertically
julien-nc Jul 3, 2024
ca3e092
start moving buttons in text field
julien-nc Jul 3, 2024
519d432
update npm pkgs
julien-nc Jul 3, 2024
5c10dc4
fix psalm and openapi issues
julien-nc Jul 3, 2024
aed6282
absolute position of copy and chooseFile buttons in text fields
julien-nc Jul 4, 2024
088049e
fix more psalm issues and update baseline to ignore remaining ones
julien-nc Jul 4, 2024
0303112
change style of number field, remove task type name+desc, add title t…
julien-nc Jul 4, 2024
4d2e33d
remove result title in form, add desc as title for text fields
julien-nc Jul 4, 2024
eb4be77
upgrade to nc/vue 8.14.0
julien-nc Jul 4, 2024
f1a3885
update sylelint, adjust NcModal for latest nc/vue
julien-nc Jul 4, 2024
28f3fe5
improve media field style, allow to clear its value
julien-nc Jul 5, 2024
ff8c052
change how to toggle advanced fields: NcActions next to submit button
julien-nc Jul 5, 2024
be4287c
improve style of media list
julien-nc Jul 5, 2024
9c37e00
fix border radius of images in task history list item
julien-nc Jul 5, 2024
061a7ee
disable debug in assistant service
julien-nc Jul 5, 2024
eeb4955
fix identifier being replaced by customId, use task appId to only sho…
julien-nc Jul 8, 2024
47b73a9
make chattyUI message/title generations asynchronous, the frontend po…
julien-nc Jul 8, 2024
306fb1d
fix input field style in chattyUI
julien-nc Jul 8, 2024
555f9d2
fix field validity check to disable the submit button
julien-nc Jul 9, 2024
2c52e3b
cs:fix, fix psalm issues
julien-nc Jul 9, 2024
75273b0
fix bug when opening the assistant and the last selected task type is…
julien-nc Jul 9, 2024
3dda242
add migration step to remove old tables
julien-nc Jul 9, 2024
30ad18a
fix data folder creation, fix input file upload: avoid name collision…
julien-nc Jul 10, 2024
0ba20a0
refactor audio recorder
julien-nc Jul 10, 2024
a17b0ce
start implementing list of text
julien-nc Jul 10, 2024
5cdd35f
start to adjust context chat custom UI to the new task processing pro…
julien-nc Jul 10, 2024
a23b3cf
finalize context chat custom UI, change how inputs are reset on taskT…
julien-nc Jul 11, 2024
1321b10
filter out the chat task type
julien-nc Jul 11, 2024
0229883
more design adjustments
julien-nc Jul 11, 2024
c01dfe4
more design adjustments
julien-nc Jul 11, 2024
af21951
improve media field UX, disable buttons when uploading/recording, cha…
julien-nc Jul 12, 2024
a62eaf3
adjust list of media field to match media field
julien-nc Jul 12, 2024
7d30404
move text input label outside, fix copy button overlapping text
julien-nc Jul 12, 2024
ad4c13b
change delete, redo, copy icons
julien-nc Jul 12, 2024
1f9474c
fix richcontenteditable style issues, make its height smaller
julien-nc Jul 12, 2024
e0df3f8
add loading icon when chat is thinking, fix conversation header style
julien-nc Jul 12, 2024
bd553c2
fix label of number field
julien-nc Jul 12, 2024
02669c6
improve canSubmit check
julien-nc Jul 12, 2024
39ab774
add type validation in number field
julien-nc Jul 12, 2024
5041364
attempt to fix issue when no task type is selected
julien-nc Jul 13, 2024
4aa7c11
nothing
julien-nc Jul 13, 2024
52611cb
polish
julien-nc Jul 15, 2024
c93f43e
generate OpenAPI specs
julien-nc Jul 15, 2024
74958af
more design adjustments
julien-nc Jul 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 2 additions & 10 deletions appinfo/info.xml
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ Known providers:
* [OpenAi/LocalAI integration](https://apps.nextcloud.com/apps/integration_openai)
* [Local Whisper Speech-To-Text](https://apps.nextcloud.com/apps/stt_whisper)
]]> </description>
<version>1.1.0</version>
<version>2.0.1</version>
<licence>agpl</licence>
<author>Julien Veyssier</author>
<namespace>Assistant</namespace>
Expand All @@ -70,16 +70,8 @@ Known providers:
<screenshot>https://github.com/nextcloud/assistant/raw/main/img/screenshot2.jpg</screenshot>
<screenshot>https://github.com/nextcloud/assistant/raw/main/img/screenshot3.jpg</screenshot>
<screenshot>https://github.com/nextcloud/assistant/raw/main/img/screenshot4.jpg</screenshot>
<background-jobs>
<job>OCA\Assistant\Cron\CleanupImageGenerations</job>
<job>OCA\Assistant\Cron\CleanupAssistantTasks</job>
</background-jobs>
<commands>
<command>OCA\Assistant\Command\CleanupImageGenerations</command>
<command>OCA\Assistant\Command\CleanupAssistantTasks</command>
</commands>
<dependencies>
<nextcloud min-version="28" max-version="30"/>
<nextcloud min-version="30" max-version="30"/>
</dependencies>
<settings>
<admin>OCA\Assistant\Settings\Admin</admin>
Expand Down
36 changes: 9 additions & 27 deletions appinfo/routes.php
Original file line number Diff line number Diff line change
Expand Up @@ -10,19 +10,9 @@
['name' => 'config#setConfig', 'url' => '/config', 'verb' => 'PUT'],
['name' => 'config#setAdminConfig', 'url' => '/admin-config', 'verb' => 'PUT'],

['name' => 'assistant#getAssistantTaskResultPage', 'url' => '/task/view/{metaTaskId}', 'verb' => 'GET'],
['name' => 'assistant#getAssistantTaskResultPage', 'url' => '/task/view/{taskId}', 'verb' => 'GET'],
['name' => 'assistant#getAssistantStandalonePage', 'url' => '/', 'verb' => 'GET'],

['name' => 'Text2Image#showGenerationPage', 'url' => '/i/{imageGenId}', 'verb' => 'GET'],
['name' => 'Text2Image#getPromptHistory', 'url' => '/i/data/prompt_history', 'verb' => 'GET'],

['name' => 'FreePrompt#processPrompt', 'url' => '/f/process_prompt', 'verb' => 'POST'],
['name' => 'FreePrompt#getPromptHistory', 'url' => '/f/prompt_history', 'verb' => 'GET'],
['name' => 'FreePrompt#getOutputs', 'url' => '/f/get_outputs', 'verb' => 'GET'],
['name' => 'FreePrompt#cancelGeneration', 'url' => '/f/cancel_generation', 'verb' => 'POST'],

['name' => 'SpeechToText#getResultPage', 'url' => '/stt/result-page/{metaTaskId}', 'verb' => 'GET'],

['name' => 'preview#getFileImage', 'url' => '/preview', 'verb' => 'GET'],

['name' => 'chattyLLM#newSession', 'url' => '/chat/new_session', 'verb' => 'PUT'],
Expand All @@ -34,27 +24,19 @@
['name' => 'chattyLLM#getMessages', 'url' => '/chat/messages', 'verb' => 'GET'],
['name' => 'chattyLLM#generateForSession', 'url' => '/chat/generate', 'verb' => 'GET'],
['name' => 'chattyLLM#regenerateForSession', 'url' => '/chat/regenerate', 'verb' => 'GET'],
['name' => 'chattyLLM#checkMessageGenerationTask', 'url' => '/chat/check_generation', 'verb' => 'GET'],
['name' => 'chattyLLM#generateTitle', 'url' => '/chat/generate_title', 'verb' => 'GET'],
['name' => 'chattyLLM#checkTitleGenerationTask', 'url' => '/chat/check_title_generation', 'verb' => 'GET'],
],
'ocs' => [
['name' => 'assistantApi#getAvailableTaskTypes', 'url' => '/api/{apiVersion}/task-types', 'verb' => 'GET', 'requirements' => $requirements],
['name' => 'assistantApi#getAssistantTask', 'url' => '/api/{apiVersion}/task/{metaTaskId}', 'verb' => 'GET', 'requirements' => $requirements],
['name' => 'assistantApi#getUserTasks', 'url' => '/api/{apiVersion}/tasks', 'verb' => 'GET', 'requirements' => $requirements],
['name' => 'assistantApi#runTextProcessingTask', 'url' => '/api/{apiVersion}/task/run', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'assistantApi#scheduleTextProcessingTask', 'url' => '/api/{apiVersion}/task/schedule', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'assistantApi#runOrScheduleTextProcessingTask', 'url' => '/api/{apiVersion}/task/run-or-schedule', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'assistantApi#parseTextFromFile', 'url' => '/api/{apiVersion}/parse-file', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'assistantApi#deleteTask', 'url' => '/api/{apiVersion}/task/{metaTaskId}', 'verb' => 'DELETE', 'requirements' => $requirements],
['name' => 'assistantApi#cancelTask', 'url' => '/api/{apiVersion}/task/cancel/{metaTaskId}', 'verb' => 'PUT', 'requirements' => $requirements],

['name' => 'Text2ImageApi#processPrompt', 'url' => '/api/{apiVersion}/i/process_prompt', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'Text2ImageApi#getGenerationInfo', 'url' => '/api/{apiVersion}/i/info/{imageGenId}', 'verb' => 'GET', 'requirements' => $requirements],
['name' => 'Text2ImageApi#getImage', 'url' => '/api/{apiVersion}/i/{imageGenId}/{fileNameId}', 'verb' => 'GET', 'requirements' => $requirements],
['name' => 'Text2ImageApi#cancelGeneration', 'url' => '/api/{apiVersion}/i/cancel_generation', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'Text2ImageApi#setVisibilityOfImageFiles', 'url' => '/api/{apiVersion}/i/visibility/{imageGenId}', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'Text2ImageApi#notifyWhenReady', 'url' => '/api/{apiVersion}/i/notify/{imageGenId}', 'verb' => 'POST', 'requirements' => $requirements],

['name' => 'SpeechToTextApi#transcribeAudio', 'url' => '/api/{apiVersion}/stt/transcribeAudio', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'SpeechToTextApi#transcribeFile', 'url' => '/api/{apiVersion}/stt/transcribeFile', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'assistantApi#notifyWhenReady', 'url' => '/api/{apiVersion}/task/{ocpTaskId}/notify', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'assistantApi#uploadInputFile', 'url' => '/api/{apiVersion}/input-file', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'assistantApi#displayUserFile', 'url' => '/api/{apiVersion}/file/{fileId}/display', 'verb' => 'GET', 'requirements' => $requirements],
['name' => 'assistantApi#getUserFileInfo', 'url' => '/api/{apiVersion}/file/{fileId}/info', 'verb' => 'GET', 'requirements' => $requirements],
['name' => 'assistantApi#shareOutputFile', 'url' => '/api/{apiVersion}/task/{ocpTaskId}/file/{fileId}/share', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'assistantApi#getOutputFilePreview', 'url' => '/api/{apiVersion}/task/{ocpTaskId}/output-file/{fileId}/preview', 'verb' => 'GET', 'requirements' => $requirements],
],
];
2 changes: 2 additions & 0 deletions composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@
"cs:check": "php-cs-fixer fix --dry-run --diff",
"cs:fix": "php-cs-fixer fix",
"psalm": "psalm.phar --no-cache",
"psalm:update-baseline": "psalm.phar --threads=1 --update-baseline",
"psalm:update-baseline:force": "psalm.phar --threads=1 --update-baseline --set-baseline=tests/psalm-baseline.xml",
"test:unit": "phpunit --config tests/phpunit.xml",
"openapi": "generate-spec --verbose"
},
Expand Down
10 changes: 5 additions & 5 deletions composer.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

42 changes: 5 additions & 37 deletions lib/AppInfo/Application.php
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,9 @@
use OCA\Assistant\Listener\BeforeTemplateRenderedListener;
use OCA\Assistant\Listener\FreePrompt\FreePromptReferenceListener;
use OCA\Assistant\Listener\SpeechToText\SpeechToTextReferenceListener;
use OCA\Assistant\Listener\SpeechToText\SpeechToTextResultListener;
use OCA\Assistant\Listener\TaskFailedListener;
use OCA\Assistant\Listener\TaskSuccessfulListener;
use OCA\Assistant\Listener\Text2Image\Text2ImageReferenceListener;
use OCA\Assistant\Listener\Text2Image\Text2ImageResultListener;
use OCA\Assistant\Notification\Notifier;
use OCA\Assistant\Reference\FreePromptReferenceProvider;
use OCA\Assistant\Reference\SpeechToTextReferenceProvider;
Expand All @@ -22,13 +20,8 @@
use OCP\AppFramework\Bootstrap\IRegistrationContext;
use OCP\AppFramework\Http\Events\BeforeTemplateRenderedEvent;
use OCP\Collaboration\Reference\RenderReferenceEvent;
use OCP\SpeechToText\Events\TranscriptionFailedEvent;
use OCP\SpeechToText\Events\TranscriptionSuccessfulEvent;
use OCP\TextProcessing\Events\TaskFailedEvent as TextTaskFailedEvent;
use OCP\TextProcessing\Events\TaskSuccessfulEvent as TextTaskSuccessfulEvent;
use OCP\TextProcessing\Task as OCPTextprocessingTask;
use OCP\TextToImage\Events\TaskFailedEvent as TextToImageTaskFailedEvent;
use OCP\TextToImage\Events\TaskSuccessfulEvent as TextToImageTaskSuccessfulEvent;
use OCP\TaskProcessing\Events\TaskFailedEvent;
use OCP\TaskProcessing\Events\TaskSuccessfulEvent;

class Application extends App implements IBootstrap {

Expand All @@ -41,27 +34,7 @@ class Application extends App implements IBootstrap {
public const DEFAULT_TEXT_GENERATION_STORAGE_TIME = 60 * 60 * 24 * 90; // 90 days
public const IMAGE_FOLDER = 'generated_images';
public const SPEECH_TO_TEXT_REC_FOLDER = 'stt_recordings';

public const STT_TASK_SCHEDULED = 0;
public const STT_TASK_SUCCESSFUL = 1;
public const STT_TASK_FAILED = -1;

public const STATUS_META_TASK_UNKNOWN = 0;
public const STATUS_META_TASK_SCHEDULED = 1;
public const STATUS_META_TASK_RUNNING = 2;
public const STATUS_META_TASK_SUCCESSFUL = 3;
public const STATUS_META_TASK_FAILED = 4;
public const TP_STATUS_TO_META_STATUS = [
OCPTextprocessingTask::STATUS_UNKNOWN => self::STATUS_META_TASK_UNKNOWN,
OCPTextprocessingTask::STATUS_SCHEDULED => self::STATUS_META_TASK_SCHEDULED,
OCPTextprocessingTask::STATUS_RUNNING => self::STATUS_META_TASK_RUNNING,
OCPTextprocessingTask::STATUS_SUCCESSFUL => self::STATUS_META_TASK_SUCCESSFUL,
OCPTextprocessingTask::STATUS_FAILED => self::STATUS_META_TASK_FAILED,
];

public const TASK_CATEGORY_TEXT_GEN = 0;
public const TASK_CATEGORY_TEXT_TO_IMAGE = 1;
public const TASK_CATEGORY_SPEECH_TO_TEXT = 2;
public const ASSISTANT_DATA_FOLDER_NAME = 'Assistant';

public const CHAT_USER_INSTRUCTIONS = 'This is a conversation in a specific language between {user} and you, Nextcloud Assistant. You are a kind, polite and helpful AI that helps {user} to the best of its abilities. If you do not understand something, you will ask for clarification. Detect the language that {user} is using. Make sure to use the same language in your response. Do not mention the language explicitly.';
public const CHAT_USER_INSTRUCTIONS_TITLE = 'Above is a chat session in a specific language between {user} and you, Nextcloud Assistant. Generate a suitable title summarizing the conversation in the same language. Output only the title in plain text, nothing else.';
Expand All @@ -83,15 +56,10 @@ public function register(IRegistrationContext $context): void {
$context->registerEventListener(RenderReferenceEvent::class, FreePromptReferenceListener::class);
$context->registerEventListener(RenderReferenceEvent::class, SpeechToTextReferenceListener::class);

$context->registerEventListener(TextToImageTaskSuccessfulEvent::class, Text2ImageResultListener::class);
$context->registerEventListener(TextToImageTaskFailedEvent::class, Text2ImageResultListener::class);
$context->registerEventListener(TranscriptionSuccessfulEvent::class, SpeechToTextResultListener::class);
$context->registerEventListener(TranscriptionFailedEvent::class, SpeechToTextResultListener::class);

$context->registerEventListener(BeforeTemplateRenderedEvent::class, BeforeTemplateRenderedListener::class);

$context->registerEventListener(TextTaskSuccessfulEvent::class, TaskSuccessfulListener::class);
$context->registerEventListener(TextTaskFailedEvent::class, TaskFailedListener::class);
$context->registerEventListener(TaskSuccessfulEvent::class, TaskSuccessfulListener::class);
$context->registerEventListener(TaskFailedEvent::class, TaskFailedListener::class);

$context->registerNotifierService(Notifier::class);
}
Expand Down
54 changes: 0 additions & 54 deletions lib/Command/CleanupAssistantTasks.php

This file was deleted.

65 changes: 0 additions & 65 deletions lib/Command/CleanupImageGenerations.php

This file was deleted.

Loading
Loading