Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check all task types availability and improve app description #46

Merged
merged 4 commits into from
Mar 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 22 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,25 +2,29 @@

This app brings a user interface to use the Nextcloud text processing feature.

It allows users to launch text processing tasks, be notified when they finish and see the results.
It allows users to launch AI tasks, be notified when they finish and see the results.
The assistant also appears in others apps like Text to easily process parts of a document.

### How to use it

A new right header menu entry appears. Once clicked, the assistant is displayed and you can select and task type and
set the input text you want to process.
A new right header menu entry appears. Once clicked, the assistant is displayed and you can select and task type and
set the input you want to process.

The task might run immediately or be scheduled depending on the time estimation given by the AI provider.
Once a task is scheduled, it will run as a background job. When it is finished, you will receive a notification
from which the results can be displayed.

Other apps can integrate with the assistant. For example, Text will display an inline button besides every paragraph
to directly select a task type to process this paragraph. Selecting a task this way will open the assistant with the task
being pre-selected and the input text set.

### Text processing providers
## Features

In the assistant, the list of available tasks depends on the available providers installed via other apps.
This means you have complete freedom over which service/software will actually run your text processing tasks.
This means you have complete freedom over which service/software will actually run your AI tasks.

### Text processing

So far, the [Large language model](https://github.com/nextcloud/llm#readme)
and the [OpenAi/LocalAI integration](https://apps.nextcloud.com/apps/integration_openai) apps
include text processing providers to:
Expand All @@ -29,3 +33,16 @@ include text processing providers to:
* Generate a headline
* Get an answer from a free prompt
* Reformulate (OpenAi/LocalAi only)
* Context writer: Generate text with a specified style. The style can be described or provided via an example text.

### Text to image (Image generation)

Known providers:
* [OpenAi/LocalAI integration](https://apps.nextcloud.com/apps/integration_openai)
* [Text2Image Stable Diffusion](https://apps.nextcloud.com/apps/text2image_stablediffusion)

### Speech to text (Audio transcription)

Known providers:
* [OpenAi/LocalAI integration](https://apps.nextcloud.com/apps/integration_openai)
* [Local Whisper Speech-To-Text](https://apps.nextcloud.com/apps/stt_whisper)
27 changes: 22 additions & 5 deletions appinfo/info.xml
Original file line number Diff line number Diff line change
Expand Up @@ -6,25 +6,29 @@
<description><![CDATA[
This app brings a user interface to use the Nextcloud text processing feature.

It allows users to launch text processing tasks, be notified when they finish and see the results.
It allows users to launch AI tasks, be notified when they finish and see the results.
The assistant also appears in others apps like Text to easily process parts of a document.

### How to use it

A new right header menu entry appears. Once clicked, the assistant is displayed and you can select and task type and
set the input text you want to process.
A new right header menu entry appears. Once clicked, the assistant is displayed and you can select and task type and
set the input you want to process.

The task might run immediately or be scheduled depending on the time estimation given by the AI provider.
Once a task is scheduled, it will run as a background job. When it is finished, you will receive a notification
from which the results can be displayed.

Other apps can integrate with the assistant. For example, Text will display an inline button besides every paragraph
to directly select a task type to process this paragraph. Selecting a task this way will open the assistant with the task
being pre-selected and the input text set.

### Text processing providers
## Features

In the assistant, the list of available tasks depends on the available providers installed via other apps.
This means you have complete freedom over which service/software will actually run your text processing tasks.
This means you have complete freedom over which service/software will actually run your AI tasks.

### Text processing providers

So far, the [Large language model](https://github.com/nextcloud/llm#readme)
and the [OpenAi/LocalAI integration](https://apps.nextcloud.com/apps/integration_openai) apps
include text processing providers to:
Expand All @@ -33,6 +37,19 @@ include text processing providers to:
* Generate a headline
* Get an answer from a free prompt
* Reformulate (OpenAi/LocalAi only)
* Context writer: Generate text with a specified style. The style can be described or provided via an example text.

### Text to image (Image generation)

Known providers:
* [OpenAi/LocalAI integration](https://apps.nextcloud.com/apps/integration_openai)
* [Text2Image Stable Diffusion](https://apps.nextcloud.com/apps/text2image_stablediffusion)

### Speech to text (Audio transcription)

Known providers:
* [OpenAi/LocalAI integration](https://apps.nextcloud.com/apps/integration_openai)
* [Local Whisper Speech-To-Text](https://apps.nextcloud.com/apps/stt_whisper)
]]> </description>
<version>1.0.5</version>
<licence>agpl</licence>
Expand Down
45 changes: 27 additions & 18 deletions appinfo/routes.php
Original file line number Diff line number Diff line change
@@ -1,37 +1,46 @@
<?php

$requirements = [
'apiVersion' => '(v1)',
];

return [
'routes' => [
['name' => 'config#getConfigValue', 'url' => '/config', 'verb' => 'GET'],
['name' => 'config#setConfig', 'url' => '/config', 'verb' => 'PUT'],
['name' => 'config#setAdminConfig', 'url' => '/admin-config', 'verb' => 'PUT'],

['name' => 'assistant#getAssistantTaskResultPage', 'url' => '/task/view/{metaTaskId}', 'verb' => 'GET'],
['name' => 'assistant#getAssistantTask', 'url' => '/task/{metaTaskId}', 'verb' => 'GET'],
['name' => 'assistant#getUserTasks', 'url' => '/tasks', 'verb' => 'GET'],
['name' => 'assistant#runTextProcessingTask', 'url' => '/task/run', 'verb' => 'POST'],
['name' => 'assistant#scheduleTextProcessingTask', 'url' => '/task/schedule', 'verb' => 'POST'],
['name' => 'assistant#runOrScheduleTextProcessingTask', 'url' => '/task/run-or-schedule', 'verb' => 'POST'],
['name' => 'assistant#parseTextFromFile', 'url' => '/parse-file', 'verb' => 'POST'],
['name' => 'assistant#deleteTask', 'url' => '/task/{metaTaskId}', 'verb' => 'DELETE'],
['name' => 'assistant#cancelTask', 'url' => '/task/cancel/{metaTaskId}', 'verb' => 'PUT'],

['name' => 'Text2Image#processPrompt', 'url' => '/i/process_prompt', 'verb' => 'POST'],
['name' => 'Text2Image#getPromptHistory', 'url' => '/i/prompt_history', 'verb' => 'GET'],

['name' => 'Text2Image#showGenerationPage', 'url' => '/i/{imageGenId}', 'verb' => 'GET'],
['name' => 'Text2Image#getGenerationInfo', 'url' => '/i/info/{imageGenId}', 'verb' => 'GET'],
['name' => 'Text2Image#getImage', 'url' => '/i/{imageGenId}/{fileNameId}', 'verb' => 'GET'],
['name' => 'Text2Image#cancelGeneration', 'url' => '/i/cancel_generation', 'verb' => 'POST'],
['name' => 'Text2Image#setVisibilityOfImageFiles', 'url' => '/i/visibility/{imageGenId}', 'verb' => 'POST'],
['name' => 'Text2Image#notifyWhenReady', 'url' => '/i/notify/{imageGenId}', 'verb' => 'POST'],

['name' => 'FreePrompt#processPrompt', 'url' => '/f/process_prompt', 'verb' => 'POST'],
['name' => 'FreePrompt#getPromptHistory', 'url' => '/f/prompt_history', 'verb' => 'GET'],
['name' => 'FreePrompt#getOutputs', 'url' => '/f/get_outputs', 'verb' => 'GET'],
['name' => 'FreePrompt#cancelGeneration', 'url' => '/f/cancel_generation', 'verb' => 'POST'],

['name' => 'SpeechToText#getResultPage', 'url' => '/stt/result-page/{metaTaskId}', 'verb' => 'GET'],
['name' => 'SpeechToText#transcribeAudio', 'url' => '/stt/transcribeAudio', 'verb' => 'POST'],
['name' => 'SpeechToText#transcribeFile', 'url' => '/stt/transcribeFile', 'verb' => 'POST'],
],
'ocs' => [
['name' => 'assistantApi#getAvailableTaskTypes', 'url' => '/api/{apiVersion}/task-types', 'verb' => 'GET', 'requirements' => $requirements],
['name' => 'assistantApi#getAssistantTask', 'url' => '/api/{apiVersion}/task/{metaTaskId}', 'verb' => 'GET', 'requirements' => $requirements],
['name' => 'assistantApi#getUserTasks', 'url' => '/api/{apiVersion}/tasks', 'verb' => 'GET', 'requirements' => $requirements],
['name' => 'assistantApi#runTextProcessingTask', 'url' => '/api/{apiVersion}/task/run', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'assistantApi#scheduleTextProcessingTask', 'url' => '/api/{apiVersion}/task/schedule', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'assistantApi#runOrScheduleTextProcessingTask', 'url' => '/api/{apiVersion}/task/run-or-schedule', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'assistantApi#parseTextFromFile', 'url' => '/api/{apiVersion}/parse-file', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'assistantApi#deleteTask', 'url' => '/api/{apiVersion}/task/{metaTaskId}', 'verb' => 'DELETE', 'requirements' => $requirements],
['name' => 'assistantApi#cancelTask', 'url' => '/api/{apiVersion}/task/cancel/{metaTaskId}', 'verb' => 'PUT', 'requirements' => $requirements],

['name' => 'Text2ImageApi#processPrompt', 'url' => '/api/{apiVersion}/i/process_prompt', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'Text2ImageApi#getPromptHistory', 'url' => '/api/{apiVersion}/i/prompt_history', 'verb' => 'GET', 'requirements' => $requirements],
['name' => 'Text2ImageApi#getGenerationInfo', 'url' => '/api/{apiVersion}/i/info/{imageGenId}', 'verb' => 'GET', 'requirements' => $requirements],
['name' => 'Text2ImageApi#getImage', 'url' => '/api/{apiVersion}/i/{imageGenId}/{fileNameId}', 'verb' => 'GET', 'requirements' => $requirements],
['name' => 'Text2ImageApi#cancelGeneration', 'url' => '/api/{apiVersion}/i/cancel_generation', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'Text2ImageApi#setVisibilityOfImageFiles', 'url' => '/api/{apiVersion}/i/visibility/{imageGenId}', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'Text2ImageApi#notifyWhenReady', 'url' => '/api/{apiVersion}/i/notify/{imageGenId}', 'verb' => 'POST', 'requirements' => $requirements],

['name' => 'SpeechToTextApi#transcribeAudio', 'url' => '/api/{apiVersion}/stt/transcribeAudio', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'SpeechToTextApi#transcribeFile', 'url' => '/api/{apiVersion}/stt/transcribeFile', 'verb' => 'POST', 'requirements' => $requirements],
],
];
195 changes: 195 additions & 0 deletions lib/Controller/AssistantApiController.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,195 @@
<?php

namespace OCA\TpAssistant\Controller;

use OCA\TpAssistant\Db\MetaTask;
use OCA\TpAssistant\Db\MetaTaskMapper;
use OCA\TpAssistant\Service\AssistantService;
use OCP\AppFramework\Http;
use OCP\AppFramework\Http\Attribute\NoAdminRequired;
use OCP\AppFramework\Http\Attribute\NoCSRFRequired;
use OCP\AppFramework\Http\DataResponse;
use OCP\AppFramework\OCSController;
use OCP\DB\Exception;
use OCP\IRequest;

class AssistantApiController extends OCSController {

public function __construct(
string $appName,
IRequest $request,
private AssistantService $assistantService,
private MetaTaskMapper $metaTaskMapper,
private ?string $userId,
) {
parent::__construct($appName, $request);
}

/**
* @return DataResponse
*/
#[NoAdminRequired]
#[NoCSRFRequired]
public function getAvailableTaskTypes(): DataResponse {
$taskTypes = $this->assistantService->getAvailableTaskTypes();
return new DataResponse(['types' => $taskTypes]);
}

/**
* @param int $metaTaskId
* @return DataResponse
*/
#[NoAdminRequired]
public function deleteTask(int $metaTaskId): DataResponse {
if ($this->userId !== null) {
try {
$this->assistantService->deleteAssistantTask($this->userId, $metaTaskId);
return new DataResponse('');
} catch (\Exception $e) {
}
}

return new DataResponse('', Http::STATUS_NOT_FOUND);
}

/**
* @param int $metaTaskId
* @return DataResponse
*/
#[NoAdminRequired]
public function cancelTask(int $metaTaskId): DataResponse {
if ($this->userId !== null) {
try {
$this->assistantService->cancelAssistantTask($this->userId, $metaTaskId);
return new DataResponse('');
} catch (\Exception $e) {
}
}

return new DataResponse('', Http::STATUS_NOT_FOUND);
}

/**
* @param int $metaTaskId
* @return DataResponse
*/
#[NoAdminRequired]
public function getAssistantTask(int $metaTaskId): DataResponse {
if ($this->userId !== null) {
$task = $this->assistantService->getAssistantTask($this->userId, $metaTaskId);
if ($task !== null) {
return new DataResponse([
'task' => $task->jsonSerializeCc(),
]);
}
}
return new DataResponse('', Http::STATUS_NOT_FOUND);
}

#[NoAdminRequired]
public function getUserTasks(?string $taskType = null, ?int $category = null): DataResponse {
if ($this->userId !== null) {
try {
$tasks = $this->metaTaskMapper->getUserMetaTasks($this->userId, $taskType, $category);
$serializedTasks = array_map(static function (MetaTask $task) {
return $task->jsonSerializeCc();
}, $tasks);
return new DataResponse(['tasks' => $serializedTasks]);
} catch (Exception $e) {
return new DataResponse(['tasks' => []]);
}
}
return new DataResponse('', Http::STATUS_NOT_FOUND);
}

/**
* @param array $inputs
* @param string $type
* @param string $appId
* @param string $identifier
* @return DataResponse
*/
#[NoAdminRequired]
public function runTextProcessingTask(string $type, array $inputs, string $appId, string $identifier): DataResponse {
if ($this->userId === null) {
return new DataResponse('Unknow user', Http::STATUS_BAD_REQUEST);
}

try {
$task = $this->assistantService->runTextProcessingTask($type, $inputs, $appId, $this->userId, $identifier);
} catch (\Exception | \Throwable $e) {
return new DataResponse($e->getMessage(), Http::STATUS_BAD_REQUEST);
}
return new DataResponse([
'task' => $task->jsonSerializeCc(),
]);
}

/**
* @param array $inputs
* @param string $type
* @param string $appId
* @param string $identifier
* @return DataResponse
*/
#[NoAdminRequired]
public function scheduleTextProcessingTask(string $type, array $inputs, string $appId, string $identifier): DataResponse {
if ($this->userId === null) {
return new DataResponse('Unknow user', Http::STATUS_BAD_REQUEST);
}

try {
$task = $this->assistantService->scheduleTextProcessingTask($type, $inputs, $appId, $this->userId, $identifier);
} catch (\Exception | \Throwable $e) {
return new DataResponse($e->getMessage(), Http::STATUS_BAD_REQUEST);
}
return new DataResponse([
'task' => $task->jsonSerializeCc(),
]);
}

/**
* @param array $inputs
* @param string $type
* @param string $appId
* @param string $identifier
* @return DataResponse
*/
#[NoAdminRequired]
public function runOrScheduleTextProcessingTask(string $type, array $inputs, string $appId, string $identifier): DataResponse {
if ($this->userId === null) {
return new DataResponse('Unknow user', Http::STATUS_BAD_REQUEST);
}

try {
$task = $this->assistantService->runOrScheduleTextProcessingTask($type, $inputs, $appId, $this->userId, $identifier);
} catch (\Exception | \Throwable $e) {
return new DataResponse($e->getMessage(), Http::STATUS_BAD_REQUEST);
}
return new DataResponse([
'task' => $task->jsonSerializeCc(),
]);
}

/**
* Parse text from file (if parsing the file type is supported)
*
* @param string $filePath
* @return DataResponse
*/
#[NoAdminRequired]
public function parseTextFromFile(string $filePath): DataResponse {
if ($this->userId === null) {
return new DataResponse('Unknow user', Http::STATUS_BAD_REQUEST);
}

try {
$text = $this->assistantService->parseTextFromFile($filePath, $this->userId);
} catch (\Exception | \Throwable $e) {
return new DataResponse($e->getMessage(), Http::STATUS_BAD_REQUEST);
}
return new DataResponse([
'parsedText' => $text,
]);
}
}
Loading
Loading