Skip to content
This repository has been archived by the owner on Jan 3, 2023. It is now read-only.

Investigate stripedReadThreadpoolSize's impact on the performance of UnEC actions #1990

Open
PHILO-HE opened this issue Oct 29, 2018 · 1 comment
Assignees

Comments

@PHILO-HE
Copy link
Member

PHILO-HE commented Oct 29, 2018

The default value for stripedReadThreadpoolSize is 18, which can be easily reached for large number of files. If no thread is available in the thread pool, the read task will be rejected but be executed by the current thread and a log is reported: "StripedRead- task is rejected by ThreadPoolExecutor. Executing it in current thread." The impact on performance will be investigated. Maybe the property should be set according to "smart.cmdlet.executors" and EC schema.

@PHILO-HE PHILO-HE self-assigned this Oct 29, 2018
@PHILO-HE
Copy link
Member Author

PHILO-HE commented Nov 1, 2018

I noticed that a method called initThreadsNumForStripedReads in DFSClient can be used to set threads num in this scenario.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant