-
Notifications
You must be signed in to change notification settings - Fork 22
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
merge to stable-23-3: filestore - LargeDeletionMarkers-related follow…
…up work, TFileRingBuffer, some small fixes and enhancements (#2183) * unstable-process: logging open sockets upon process start failure (#2048) * issue-1541: implemented a ring buffer over a file to use it in HandleOpsQueue (#2047) * issue-1541: ring buffer over a file * issue-1541: TFileRingBufferTest::Should{Restore,Validate} + fixes * issue-1541: randomized test for TFileRingBuffer + fixes * issue-1541: moved TFileRingBuffer implementation to .cpp * issue-1541: minor rename * issue-1541: fixed ubsan (#2087) * NBS and Filestore cluster deployment instructions (#2094) * issue-1795: LargeDeletionMarkers - simple integration test (#2122) * fixed StatFs diagnostics RequestType (#2127) * issue-1795: backpressure for LargeDeletionMarkers (#2147) * issue-1795: LargeDeletionMarkers backpressure - some fixes and cleanup (#2160) * filestore-client: fixed crashes on DestroySession error, fixed ls node type output (#2168) * filestore-client: fixed crashes on DestroySession error, fixed ls node type output * filestore-client: fixed crashes on DestroySession error, fixed ls node type output - cleanup * issue-1795: implemented Unsafe{Delete,Update,Get}Node to use it in private api for manual interventions - e.g. to clean up OrphanNodes (#2165) * issue-1795: implemented Unsafe{Delete,Update,Get}Node to use it in private api for manual interventions - e.g. to clean up OrphanNodes * issue-1795: tiny cleanup * issue-1795: private api Unsafe{Delete,Update,Get}Node actions + ut * issue-1795: cleanup * updated CMakeLists.txt after merge, fixed includes * fixed CMakeLists.txt
- Loading branch information
Showing
171 changed files
with
8,353 additions
and
111 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,13 +1,16 @@ | ||
| ## Network Block Store | ||
| ## Network Block Store and Network File Store | ||
|
|
||
| Network Block Device implementation over YDB BlobStorage or over our own storage nodes. Offers reliable thin-provisioned block devices which support snapshots. | ||
| Network File System implementation over YDB BlobStorage. Offers POSIX-compliant scalable filesystem which can be attached to virtual machines via virtiofs or simply mounted via FUSE. | ||
|
|
||
| ### Quickstart | ||
|
|
||
| Follow the instructions [here](example/README.md) to build and run NBS on your machine and to attach an NBS-based disk via NBD. NBS-based disks can be attached via vhost-user-blk as well. | ||
|
|
||
| Follow the instructions [here](cloud/filestore/README.md) to build and run Filestore on your machine and to attach it to a virtual machine via virtiofs or mount it on your host via FUSE. | ||
|
|
||
| Follow the instructions [here](CLANG-FORMAT.md) to install clang-format for formatting the code. | ||
|
|
||
| ### How to Deploy | ||
|
|
||
| TODO | ||
| Follow the instructions [here](deploy/README.md) to deploy NBS and Filestore on a cluster. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
164 changes: 164 additions & 0 deletions
164
cloud/filestore/libs/storage/service/service_actor_actions_unsafe_node_ops.cpp
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,164 @@ | ||
| #include "service_actor.h" | ||
|
|
||
| #include <cloud/filestore/libs/storage/api/service.h> | ||
| #include <cloud/filestore/libs/storage/api/tablet.h> | ||
| #include <cloud/filestore/libs/storage/api/tablet_proxy.h> | ||
| #include <cloud/filestore/libs/storage/core/public.h> | ||
| #include <cloud/filestore/private/api/protos/tablet.pb.h> | ||
|
|
||
| #include <library/cpp/actors/core/actor_bootstrapped.h> | ||
|
|
||
| #include <google/protobuf/util/json_util.h> | ||
|
|
||
| namespace NCloud::NFileStore::NStorage { | ||
|
|
||
| using namespace NActors; | ||
|
|
||
| using namespace NKikimr; | ||
|
|
||
| namespace { | ||
|
|
||
| //////////////////////////////////////////////////////////////////////////////// | ||
|
|
||
| template <typename TRequest, typename TResponse> | ||
| class TTabletActionActor final | ||
| : public TActorBootstrapped<TTabletActionActor<TRequest, TResponse>> | ||
| { | ||
| private: | ||
| const TRequestInfoPtr RequestInfo; | ||
| const TString Input; | ||
|
|
||
| using TBase = TActorBootstrapped<TTabletActionActor<TRequest, TResponse>>; | ||
|
|
||
| public: | ||
| TTabletActionActor( | ||
| TRequestInfoPtr requestInfo, | ||
| TString input); | ||
|
|
||
| void Bootstrap(const TActorContext& ctx); | ||
|
|
||
| private: | ||
| void ReplyAndDie( | ||
| const TActorContext& ctx, | ||
| const TResponse::ProtoRecordType& responseRecord); | ||
|
|
||
| private: | ||
| STFUNC(StateWork) | ||
| { | ||
| switch (ev->GetTypeRewrite()) { | ||
| HFunc(TResponse, HandleResponse); | ||
|
|
||
| default: | ||
| HandleUnexpectedEvent(ev, TFileStoreComponents::SERVICE); | ||
| break; | ||
| } | ||
| } | ||
|
|
||
| void HandleResponse(const TResponse::TPtr& ev, const TActorContext& ctx); | ||
| }; | ||
|
|
||
| //////////////////////////////////////////////////////////////////////////////// | ||
|
|
||
| template <typename TRequest, typename TResponse> | ||
| TTabletActionActor<TRequest, TResponse>::TTabletActionActor( | ||
| TRequestInfoPtr requestInfo, | ||
| TString input) | ||
| : RequestInfo(std::move(requestInfo)) | ||
| , Input(std::move(input)) | ||
| {} | ||
|
|
||
| template <typename TRequest, typename TResponse> | ||
| void TTabletActionActor<TRequest, TResponse>::Bootstrap( | ||
| const TActorContext& ctx) | ||
| { | ||
| typename TRequest::ProtoRecordType request; | ||
| if (!google::protobuf::util::JsonStringToMessage(Input, &request).ok()) { | ||
| ReplyAndDie( | ||
| ctx, | ||
| TErrorResponse(E_ARGUMENT, "Failed to parse input")); | ||
| return; | ||
| } | ||
|
|
||
| if (!request.GetFileSystemId()) { | ||
| ReplyAndDie( | ||
| ctx, | ||
| TErrorResponse(E_ARGUMENT, "FileSystem id should be supplied")); | ||
| return; | ||
| } | ||
|
|
||
| auto requestToTablet = std::make_unique<TRequest>(); | ||
| requestToTablet->Record = std::move(request); | ||
|
|
||
| NCloud::Send( | ||
| ctx, | ||
| MakeIndexTabletProxyServiceId(), | ||
| std::move(requestToTablet)); | ||
|
|
||
| TBase::Become(&TTabletActionActor<TRequest, TResponse>::StateWork); | ||
| } | ||
|
|
||
| template <typename TRequest, typename TResponse> | ||
| void TTabletActionActor<TRequest, TResponse>::ReplyAndDie( | ||
| const TActorContext& ctx, | ||
| const TResponse::ProtoRecordType& response) | ||
| { | ||
| auto msg = std::make_unique<TEvService::TEvExecuteActionResponse>( | ||
| response.GetError()); | ||
|
|
||
| google::protobuf::util::MessageToJsonString( | ||
| response, | ||
| msg->Record.MutableOutput()); | ||
|
|
||
| NCloud::Reply(ctx, *RequestInfo, std::move(msg)); | ||
| TBase::Die(ctx); | ||
| } | ||
|
|
||
| //////////////////////////////////////////////////////////////////////////////// | ||
|
|
||
| template <typename TRequest, typename TResponse> | ||
| void TTabletActionActor<TRequest, TResponse>::HandleResponse( | ||
| const TResponse::TPtr& ev, | ||
| const TActorContext& ctx) | ||
| { | ||
| ReplyAndDie(ctx, ev->Get()->Record); | ||
| } | ||
|
|
||
| } // namespace | ||
|
|
||
| IActorPtr TStorageServiceActor::CreateUnsafeDeleteNodeActionActor( | ||
| TRequestInfoPtr requestInfo, | ||
| TString input) | ||
| { | ||
| using TUnsafeDeleteNodeActor = TTabletActionActor< | ||
| TEvIndexTablet::TEvUnsafeDeleteNodeRequest, | ||
| TEvIndexTablet::TEvUnsafeDeleteNodeResponse>; | ||
| return std::make_unique<TUnsafeDeleteNodeActor>( | ||
| std::move(requestInfo), | ||
| std::move(input)); | ||
| } | ||
|
|
||
| IActorPtr TStorageServiceActor::CreateUnsafeUpdateNodeActionActor( | ||
| TRequestInfoPtr requestInfo, | ||
| TString input) | ||
| { | ||
| using TUnsafeUpdateNodeActor = TTabletActionActor< | ||
| TEvIndexTablet::TEvUnsafeUpdateNodeRequest, | ||
| TEvIndexTablet::TEvUnsafeUpdateNodeResponse>; | ||
| return std::make_unique<TUnsafeUpdateNodeActor>( | ||
| std::move(requestInfo), | ||
| std::move(input)); | ||
| } | ||
|
|
||
| IActorPtr TStorageServiceActor::CreateUnsafeGetNodeActionActor( | ||
| TRequestInfoPtr requestInfo, | ||
| TString input) | ||
| { | ||
| using TUnsafeGetNodeActor = TTabletActionActor< | ||
| TEvIndexTablet::TEvUnsafeGetNodeRequest, | ||
| TEvIndexTablet::TEvUnsafeGetNodeResponse>; | ||
| return std::make_unique<TUnsafeGetNodeActor>( | ||
| std::move(requestInfo), | ||
| std::move(input)); | ||
| } | ||
|
|
||
| } // namespace NCloud::NFileStore::NStorage |
Oops, something went wrong.