Skip to content

Commit

Permalink
Merge pull request #46 from nextcloud/enh/check-all-task-types-availa…
Browse files Browse the repository at this point in the history
…bility

Check all task types availability and improve app description
  • Loading branch information
julien-nc committed Mar 8, 2024
2 parents 4539191 + 6bad4b0 commit 1d07caf
Show file tree
Hide file tree
Showing 22 changed files with 843 additions and 666 deletions.
27 changes: 22 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,25 +2,29 @@

This app brings a user interface to use the Nextcloud text processing feature.

It allows users to launch text processing tasks, be notified when they finish and see the results.
It allows users to launch AI tasks, be notified when they finish and see the results.
The assistant also appears in others apps like Text to easily process parts of a document.

### How to use it

A new right header menu entry appears. Once clicked, the assistant is displayed and you can select and task type and
set the input text you want to process.
A new right header menu entry appears. Once clicked, the assistant is displayed and you can select and task type and
set the input you want to process.

The task might run immediately or be scheduled depending on the time estimation given by the AI provider.
Once a task is scheduled, it will run as a background job. When it is finished, you will receive a notification
from which the results can be displayed.

Other apps can integrate with the assistant. For example, Text will display an inline button besides every paragraph
to directly select a task type to process this paragraph. Selecting a task this way will open the assistant with the task
being pre-selected and the input text set.

### Text processing providers
## Features

In the assistant, the list of available tasks depends on the available providers installed via other apps.
This means you have complete freedom over which service/software will actually run your text processing tasks.
This means you have complete freedom over which service/software will actually run your AI tasks.

### Text processing

So far, the [Large language model](https://github.com/nextcloud/llm#readme)
and the [OpenAi/LocalAI integration](https://apps.nextcloud.com/apps/integration_openai) apps
include text processing providers to:
Expand All @@ -29,3 +33,16 @@ include text processing providers to:
* Generate a headline
* Get an answer from a free prompt
* Reformulate (OpenAi/LocalAi only)
* Context writer: Generate text with a specified style. The style can be described or provided via an example text.

### Text to image (Image generation)

Known providers:
* [OpenAi/LocalAI integration](https://apps.nextcloud.com/apps/integration_openai)
* [Text2Image Stable Diffusion](https://apps.nextcloud.com/apps/text2image_stablediffusion)

### Speech to text (Audio transcription)

Known providers:
* [OpenAi/LocalAI integration](https://apps.nextcloud.com/apps/integration_openai)
* [Local Whisper Speech-To-Text](https://apps.nextcloud.com/apps/stt_whisper)
27 changes: 22 additions & 5 deletions appinfo/info.xml
Original file line number Diff line number Diff line change
Expand Up @@ -6,25 +6,29 @@
<description><![CDATA[
This app brings a user interface to use the Nextcloud text processing feature.
It allows users to launch text processing tasks, be notified when they finish and see the results.
It allows users to launch AI tasks, be notified when they finish and see the results.
The assistant also appears in others apps like Text to easily process parts of a document.
### How to use it
A new right header menu entry appears. Once clicked, the assistant is displayed and you can select and task type and
set the input text you want to process.
A new right header menu entry appears. Once clicked, the assistant is displayed and you can select and task type and
set the input you want to process.
The task might run immediately or be scheduled depending on the time estimation given by the AI provider.
Once a task is scheduled, it will run as a background job. When it is finished, you will receive a notification
from which the results can be displayed.
Other apps can integrate with the assistant. For example, Text will display an inline button besides every paragraph
to directly select a task type to process this paragraph. Selecting a task this way will open the assistant with the task
being pre-selected and the input text set.
### Text processing providers
## Features
In the assistant, the list of available tasks depends on the available providers installed via other apps.
This means you have complete freedom over which service/software will actually run your text processing tasks.
This means you have complete freedom over which service/software will actually run your AI tasks.
### Text processing providers
So far, the [Large language model](https://github.com/nextcloud/llm#readme)
and the [OpenAi/LocalAI integration](https://apps.nextcloud.com/apps/integration_openai) apps
include text processing providers to:
Expand All @@ -33,6 +37,19 @@ include text processing providers to:
* Generate a headline
* Get an answer from a free prompt
* Reformulate (OpenAi/LocalAi only)
* Context writer: Generate text with a specified style. The style can be described or provided via an example text.
### Text to image (Image generation)
Known providers:
* [OpenAi/LocalAI integration](https://apps.nextcloud.com/apps/integration_openai)
* [Text2Image Stable Diffusion](https://apps.nextcloud.com/apps/text2image_stablediffusion)
### Speech to text (Audio transcription)
Known providers:
* [OpenAi/LocalAI integration](https://apps.nextcloud.com/apps/integration_openai)
* [Local Whisper Speech-To-Text](https://apps.nextcloud.com/apps/stt_whisper)
]]> </description>
<version>1.0.5</version>
<licence>agpl</licence>
Expand Down
45 changes: 27 additions & 18 deletions appinfo/routes.php
Original file line number Diff line number Diff line change
@@ -1,37 +1,46 @@
<?php

$requirements = [
'apiVersion' => '(v1)',
];

return [
'routes' => [
['name' => 'config#getConfigValue', 'url' => '/config', 'verb' => 'GET'],
['name' => 'config#setConfig', 'url' => '/config', 'verb' => 'PUT'],
['name' => 'config#setAdminConfig', 'url' => '/admin-config', 'verb' => 'PUT'],

['name' => 'assistant#getAssistantTaskResultPage', 'url' => '/task/view/{metaTaskId}', 'verb' => 'GET'],
['name' => 'assistant#getAssistantTask', 'url' => '/task/{metaTaskId}', 'verb' => 'GET'],
['name' => 'assistant#getUserTasks', 'url' => '/tasks', 'verb' => 'GET'],
['name' => 'assistant#runTextProcessingTask', 'url' => '/task/run', 'verb' => 'POST'],
['name' => 'assistant#scheduleTextProcessingTask', 'url' => '/task/schedule', 'verb' => 'POST'],
['name' => 'assistant#runOrScheduleTextProcessingTask', 'url' => '/task/run-or-schedule', 'verb' => 'POST'],
['name' => 'assistant#parseTextFromFile', 'url' => '/parse-file', 'verb' => 'POST'],
['name' => 'assistant#deleteTask', 'url' => '/task/{metaTaskId}', 'verb' => 'DELETE'],
['name' => 'assistant#cancelTask', 'url' => '/task/cancel/{metaTaskId}', 'verb' => 'PUT'],

['name' => 'Text2Image#processPrompt', 'url' => '/i/process_prompt', 'verb' => 'POST'],
['name' => 'Text2Image#getPromptHistory', 'url' => '/i/prompt_history', 'verb' => 'GET'],

['name' => 'Text2Image#showGenerationPage', 'url' => '/i/{imageGenId}', 'verb' => 'GET'],
['name' => 'Text2Image#getGenerationInfo', 'url' => '/i/info/{imageGenId}', 'verb' => 'GET'],
['name' => 'Text2Image#getImage', 'url' => '/i/{imageGenId}/{fileNameId}', 'verb' => 'GET'],
['name' => 'Text2Image#cancelGeneration', 'url' => '/i/cancel_generation', 'verb' => 'POST'],
['name' => 'Text2Image#setVisibilityOfImageFiles', 'url' => '/i/visibility/{imageGenId}', 'verb' => 'POST'],
['name' => 'Text2Image#notifyWhenReady', 'url' => '/i/notify/{imageGenId}', 'verb' => 'POST'],

['name' => 'FreePrompt#processPrompt', 'url' => '/f/process_prompt', 'verb' => 'POST'],
['name' => 'FreePrompt#getPromptHistory', 'url' => '/f/prompt_history', 'verb' => 'GET'],
['name' => 'FreePrompt#getOutputs', 'url' => '/f/get_outputs', 'verb' => 'GET'],
['name' => 'FreePrompt#cancelGeneration', 'url' => '/f/cancel_generation', 'verb' => 'POST'],

['name' => 'SpeechToText#getResultPage', 'url' => '/stt/result-page/{metaTaskId}', 'verb' => 'GET'],
['name' => 'SpeechToText#transcribeAudio', 'url' => '/stt/transcribeAudio', 'verb' => 'POST'],
['name' => 'SpeechToText#transcribeFile', 'url' => '/stt/transcribeFile', 'verb' => 'POST'],
],
'ocs' => [
['name' => 'assistantApi#getAvailableTaskTypes', 'url' => '/api/{apiVersion}/task-types', 'verb' => 'GET', 'requirements' => $requirements],
['name' => 'assistantApi#getAssistantTask', 'url' => '/api/{apiVersion}/task/{metaTaskId}', 'verb' => 'GET', 'requirements' => $requirements],
['name' => 'assistantApi#getUserTasks', 'url' => '/api/{apiVersion}/tasks', 'verb' => 'GET', 'requirements' => $requirements],
['name' => 'assistantApi#runTextProcessingTask', 'url' => '/api/{apiVersion}/task/run', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'assistantApi#scheduleTextProcessingTask', 'url' => '/api/{apiVersion}/task/schedule', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'assistantApi#runOrScheduleTextProcessingTask', 'url' => '/api/{apiVersion}/task/run-or-schedule', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'assistantApi#parseTextFromFile', 'url' => '/api/{apiVersion}/parse-file', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'assistantApi#deleteTask', 'url' => '/api/{apiVersion}/task/{metaTaskId}', 'verb' => 'DELETE', 'requirements' => $requirements],
['name' => 'assistantApi#cancelTask', 'url' => '/api/{apiVersion}/task/cancel/{metaTaskId}', 'verb' => 'PUT', 'requirements' => $requirements],

['name' => 'Text2ImageApi#processPrompt', 'url' => '/api/{apiVersion}/i/process_prompt', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'Text2ImageApi#getPromptHistory', 'url' => '/api/{apiVersion}/i/prompt_history', 'verb' => 'GET', 'requirements' => $requirements],
['name' => 'Text2ImageApi#getGenerationInfo', 'url' => '/api/{apiVersion}/i/info/{imageGenId}', 'verb' => 'GET', 'requirements' => $requirements],
['name' => 'Text2ImageApi#getImage', 'url' => '/api/{apiVersion}/i/{imageGenId}/{fileNameId}', 'verb' => 'GET', 'requirements' => $requirements],
['name' => 'Text2ImageApi#cancelGeneration', 'url' => '/api/{apiVersion}/i/cancel_generation', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'Text2ImageApi#setVisibilityOfImageFiles', 'url' => '/api/{apiVersion}/i/visibility/{imageGenId}', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'Text2ImageApi#notifyWhenReady', 'url' => '/api/{apiVersion}/i/notify/{imageGenId}', 'verb' => 'POST', 'requirements' => $requirements],

['name' => 'SpeechToTextApi#transcribeAudio', 'url' => '/api/{apiVersion}/stt/transcribeAudio', 'verb' => 'POST', 'requirements' => $requirements],
['name' => 'SpeechToTextApi#transcribeFile', 'url' => '/api/{apiVersion}/stt/transcribeFile', 'verb' => 'POST', 'requirements' => $requirements],
],
];
195 changes: 195 additions & 0 deletions lib/Controller/AssistantApiController.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,195 @@
<?php

namespace OCA\TpAssistant\Controller;

use OCA\TpAssistant\Db\MetaTask;
use OCA\TpAssistant\Db\MetaTaskMapper;
use OCA\TpAssistant\Service\AssistantService;
use OCP\AppFramework\Http;
use OCP\AppFramework\Http\Attribute\NoAdminRequired;
use OCP\AppFramework\Http\Attribute\NoCSRFRequired;
use OCP\AppFramework\Http\DataResponse;
use OCP\AppFramework\OCSController;
use OCP\DB\Exception;
use OCP\IRequest;

class AssistantApiController extends OCSController {

public function __construct(
string $appName,
IRequest $request,
private AssistantService $assistantService,
private MetaTaskMapper $metaTaskMapper,
private ?string $userId,
) {
parent::__construct($appName, $request);
}

/**
* @return DataResponse
*/
#[NoAdminRequired]
#[NoCSRFRequired]
public function getAvailableTaskTypes(): DataResponse {
$taskTypes = $this->assistantService->getAvailableTaskTypes();
return new DataResponse(['types' => $taskTypes]);
}

/**
* @param int $metaTaskId
* @return DataResponse
*/
#[NoAdminRequired]
public function deleteTask(int $metaTaskId): DataResponse {
if ($this->userId !== null) {
try {
$this->assistantService->deleteAssistantTask($this->userId, $metaTaskId);
return new DataResponse('');
} catch (\Exception $e) {
}
}

return new DataResponse('', Http::STATUS_NOT_FOUND);
}

/**
* @param int $metaTaskId
* @return DataResponse
*/
#[NoAdminRequired]
public function cancelTask(int $metaTaskId): DataResponse {
if ($this->userId !== null) {
try {
$this->assistantService->cancelAssistantTask($this->userId, $metaTaskId);
return new DataResponse('');
} catch (\Exception $e) {
}
}

return new DataResponse('', Http::STATUS_NOT_FOUND);
}

/**
* @param int $metaTaskId
* @return DataResponse
*/
#[NoAdminRequired]
public function getAssistantTask(int $metaTaskId): DataResponse {
if ($this->userId !== null) {
$task = $this->assistantService->getAssistantTask($this->userId, $metaTaskId);
if ($task !== null) {
return new DataResponse([
'task' => $task->jsonSerializeCc(),
]);
}
}
return new DataResponse('', Http::STATUS_NOT_FOUND);
}

#[NoAdminRequired]
public function getUserTasks(?string $taskType = null, ?int $category = null): DataResponse {
if ($this->userId !== null) {
try {
$tasks = $this->metaTaskMapper->getUserMetaTasks($this->userId, $taskType, $category);
$serializedTasks = array_map(static function (MetaTask $task) {
return $task->jsonSerializeCc();
}, $tasks);
return new DataResponse(['tasks' => $serializedTasks]);
} catch (Exception $e) {
return new DataResponse(['tasks' => []]);
}
}
return new DataResponse('', Http::STATUS_NOT_FOUND);
}

/**
* @param array $inputs
* @param string $type
* @param string $appId
* @param string $identifier
* @return DataResponse
*/
#[NoAdminRequired]
public function runTextProcessingTask(string $type, array $inputs, string $appId, string $identifier): DataResponse {
if ($this->userId === null) {
return new DataResponse('Unknow user', Http::STATUS_BAD_REQUEST);
}

try {
$task = $this->assistantService->runTextProcessingTask($type, $inputs, $appId, $this->userId, $identifier);
} catch (\Exception | \Throwable $e) {
return new DataResponse($e->getMessage(), Http::STATUS_BAD_REQUEST);
}
return new DataResponse([
'task' => $task->jsonSerializeCc(),
]);
}

/**
* @param array $inputs
* @param string $type
* @param string $appId
* @param string $identifier
* @return DataResponse
*/
#[NoAdminRequired]
public function scheduleTextProcessingTask(string $type, array $inputs, string $appId, string $identifier): DataResponse {
if ($this->userId === null) {
return new DataResponse('Unknow user', Http::STATUS_BAD_REQUEST);
}

try {
$task = $this->assistantService->scheduleTextProcessingTask($type, $inputs, $appId, $this->userId, $identifier);
} catch (\Exception | \Throwable $e) {
return new DataResponse($e->getMessage(), Http::STATUS_BAD_REQUEST);
}
return new DataResponse([
'task' => $task->jsonSerializeCc(),
]);
}

/**
* @param array $inputs
* @param string $type
* @param string $appId
* @param string $identifier
* @return DataResponse
*/
#[NoAdminRequired]
public function runOrScheduleTextProcessingTask(string $type, array $inputs, string $appId, string $identifier): DataResponse {
if ($this->userId === null) {
return new DataResponse('Unknow user', Http::STATUS_BAD_REQUEST);
}

try {
$task = $this->assistantService->runOrScheduleTextProcessingTask($type, $inputs, $appId, $this->userId, $identifier);
} catch (\Exception | \Throwable $e) {
return new DataResponse($e->getMessage(), Http::STATUS_BAD_REQUEST);
}
return new DataResponse([
'task' => $task->jsonSerializeCc(),
]);
}

/**
* Parse text from file (if parsing the file type is supported)
*
* @param string $filePath
* @return DataResponse
*/
#[NoAdminRequired]
public function parseTextFromFile(string $filePath): DataResponse {
if ($this->userId === null) {
return new DataResponse('Unknow user', Http::STATUS_BAD_REQUEST);
}

try {
$text = $this->assistantService->parseTextFromFile($filePath, $this->userId);
} catch (\Exception | \Throwable $e) {
return new DataResponse($e->getMessage(), Http::STATUS_BAD_REQUEST);
}
return new DataResponse([
'parsedText' => $text,
]);
}
}
Loading

0 comments on commit 1d07caf

Please sign in to comment.