Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

An Extremely Simple and Efficient Script I Think You All Like #1308

Closed
wants to merge 12 commits into from

Conversation

emrakyz
Copy link
Contributor

@emrakyz emrakyz commented Apr 22, 2023

EDIT: Superseded by #1419

The inspiration: (Luke Smith's Video): "Unix Chad JUST WON'T STOP Piping!"

Execution Time: 0.0 seconds on my system with more than 10.000 video files.

Very Short Summary:

  • This script feeds the "dmenu" with a list of all videos in a hard drive.
  • File paths and extensions are removed for better readability for the dmenu list.
  • Opens the chosen video file in "mpv" using the complete paths and extensions.
  • An approach of having the least amount of lines & characters while keeping the simplicity and performance has been used.

The actual script is actually a 3 line script but I added some error handling to increase convenience. You can delete everything but the last 3 lines if you want.

The first 20 lines of the script checks if related programs installed, and check if you have a video database for locate command. If not it does everything for you and notifies you appropriately. If you can't run sudo commands without entering your password, then it will also notify you about it regarding that you need to run the script on terminal or change your permissions.

Detailed Explanation for Everyone:

locate command is for locating all of the video files with their given extensions:
-d ~/.config/.mymlocate.db: Specifies the path to the database to use.
-b: Matches only the name of the files (no directory names).
-r ... : Uses a regular expression to match filenames. Specifically, it's matching files that end with the given extensions.
> "$tempfile": This redirects the output (the matched file paths) of the locate command to the temporary file we created earlier.

chosen_name=$(sed 's|./||; s/.[^.]$//' "$tempfile" | dmenu -p "Select Video"):
's|.*/||' This sed expression (details below) removes the directory part of each file path, leaving just the base filename.
s/.[^.]*$// This sed expression removes the file extension from the base filename.

**chosen_path=$(fgrep "/$chosen_name." : The pattern to search for. It looks for lines that contain our chosen filename followed by a period, ensuring we match the correct file. fgrep is faster than grep because it doesn't have regex processor. It's also safer so you can open file names with regex characters.

Finally we use the chosen path with mpv.

Regular Expressions

We combine two sed substitution commands into one command by separating them with ;

We use this scheme here: 's|pattern|replacement|; s|pattern|replacement|'

's|.*/||'

s Stands for "substitute".

| This is an alternative delimiter used for the substitution operation. Traditionally, the / character is used as the delimiter, but in this case, | has been chosen likely to avoid having to escape the / character in the pattern, making it more readable.

Pattern: .*/

.* This matches any character (represented by .) zero or more times (represented by *).

/ This matches a forward slash. Since we use | for the separator we don't have to escape the slash.

So, .*/ matches everything up to the last / in a string.

For example, for the string /home/user/videos/movie.mp4, the pattern .*/ matches /home/user/videos/

Replacement: ||

There's nothing between the two | delimiters, which means it's an empty string.

This command will replace everything up to and including the last / with nothing (it'll remove that part), effectively extracting just the base filename from a full path.

's/.[^.]*$//'

We use traditional scheme here s/pattern/replacement/

s As before, this is the "substitute" command.

/ This time, the traditional / delimiter is used.

.[^.]*$

. Matches a literal dot (.). The dot is prefixed with a backslash to escape it, since a dot has special meaning in regex (it matches any character).

[^.] The square brackets denote a character class. The ^ at the beginning of this class negates the character class, meaning it will match any character that is not a dot.

* Matches the preceding pattern (in this case, the negated character class) zero or more times.

$ Matches the end of the line.

This pattern matches the last dot and everything after it (up to the end of the line) that doesn't contain another dot. It's designed to match file extensions.

For example, in the filename movie.part1.mp4, the pattern will match .mp4.

Replacement: //

Again, there's nothing between the two / delimiters, so this part will be replaced with nothing.

This command will remove the last file extension from a filename.

When you run both commands in sequence using the semicolon to separate them, you're effectively:

Removing the path, leaving only the base filename.
Removing the file extension from that filename.

For the string /home/user/videos/movie.part1.mp4, after both sed operations, you'll be left with movie.part1

The inspiration: (Luke Smith's Video: "Unix Chad JUST WON'T STOP Piping!" youtu.be/w_37upFE-qw

Execution Time: 0.15s to 0.30s on my system (opening a video then closing it to see the output of "time" command)

Very Short Summary: 
1- This script feeds the "dmenu" with a list of all videos in a hard drive.
2- File paths and extensions are removed for better readability for the dmenu list.
3- Opens the chosen video file in "mpv" using the complete paths and extensions.
4- An approach of having the least amount of lines & characters while keeping the simplicity and performance has been used.

updatedb command needs to be executed at first showing the database file "~/.config/.mymlocate.db" Or the script should be changed with a new path.
For example: $updatedb -o ~/.config/.mymlocate.db -U /mnt/externaldrive

Thoroughly detailed explanation for everyone:

temp_file=$(mktemp): This command creates a temporary file using the mktemp command and stores its path in the temp_file variable. This temporary file will be used later to store the search results.

locate -d ~/.config/.mymlocate.db -b -r '.*\.\(mp4\|mkv\...)$': This command searches for files with the ".mp4" or other extension in the database located at "~/.config/.mymlocate.db". The -d flag specifies the database path, the -b flag searches for files with matching base names (ignores the path), and the -r flag allows the use of regular expressions for matching.

awk -F/ '{name=$NF; gsub(/\.(mp4|mkv)$/, "", name); print name "\t" $0}': This command processes the search results obtained from the locate command. It uses / as a field separator and works on each line of the output:
name=$NF: Stores the last field (file name with extension) in the name variable.
gsub(/\.(mp4|mkv)$/, "", name): Removes the ".mp4" or ".mkv" file extension from the name variable.
print name "\t" $0: Prints the modified file name (without extension) followed by a tab character and the original line (complete file path with the extension).

> "$temp_file": Redirects the output of the previous awk command to the temporary file created earlier.

chosen_file_name_and_path=$(cut -f1 "$temp_file" | dmenu -i -l 10 -p "Select a video file:"): This command reads the file names without extensions from the temporary file, presents them to the user using dmenu, and stores the user's selection in the chosen_file_name_and_path variable. The cut -f1 command extracts the file names, and the dmenu -i -l 10 -p "Select a video file:" command displays the list in a menu with a prompt.

mpv "$(awk -F'\t' -v chosen="$chosen_file_name_and_path" '$1 == chosen {print $2; exit}' "$temp_file")": This command plays the selected file using mpv. It uses awk to search the temporary file for the line where the first field (file name without extension) matches the user's selection and then extracts the second field (the complete file path with the extension) to pass it as an argument to the mpv command.

&& rm "$temp_file": After the mpv command has finished executing, this command removes the temporary file.
@TheYellowArchitect
Copy link
Contributor

TheYellowArchitect commented Nov 18, 2023

updatedb command output:
sudo: updatedb: command not found ((maybe it has something to do with lingering sudo rights after installation))

Anyway, requires the installation of locate to avoid the above problem.
I suggest adding this line:
ifinstalled locate || { echo >&2 "'locate' is required but it's not installed."; exit 1; }

The command updatedb -o ~/.config/.mymlocate.db -U /mnt/externaldrive/Videos which contains around 1 TB was surprisingly fast, didn't even take a minute.

I can confirm it works as is. It does contain videos in nested folders, and it does open the selected video with mpv, but I suggest the following 3 additions:

  1. Make it case-insensitive. If some word starts with capital (youtube video names don't follow proper grammar) it doesn't show up, so I cannot find or select it
  2. Have many choices show, instead of 1 covering from the end of input to the right edge of the screen (dropdown would also be preferable)
  3. When you press Esc for dmenu input, it shouldn't output:
fgrep: warning: fgrep is obsolescent; using grep -F
[file] Cannot open file '': No such file or directory
Failed to open .

Exiting... (Errors when loading file)

Off-topic: @emrakyz Reading the lower portion of your post (3 lines of code vs walls of text), I realized it's a tutorial on how the symbols were assembled, pretty cool explanation. If you made this script by no prior knowledge of bash, and only by watching the above video (which I too used to get started), this is impressive.

@emrakyz
Copy link
Contributor Author

emrakyz commented Nov 18, 2023

You can use my other script on another pull request named "fixnames" to make your file names Unix compatible and increase your consistency. It's extremely fast.

fgrep can be changed with grep -F to remove that error but it's harmless.

This script can be made more robust in general but I prefer simplicity for jobs like this instead of extreme correctness: "Worse is Better".

And thanks but no. I didn't just watch that video and write the script :). I am very interested in C and shell scripting even though I am not a programmer myself. I try to write extremely minimalist scripts by achieving the same targets. I generally prefer Dash and I use Bash only if I feel like I have to use it.

I use a dropdown menu. I forgot that dmenu uses an option for that. I use Rofi on Wayland since dmenu is not available on Wayland (there are some forks though). I can add it for you.

@emrakyz
Copy link
Contributor Author

emrakyz commented Nov 18, 2023

@TheYellowArchitect

By the way, on Rofi, there is an option called -auto-select that selects the matching entry if it's the only one, automatically without enter like you mentioned for "@@". On Dmenu as far as I know, it's not possible but it can be made possible with patches. I can't do anything about it in the script. Shell scripting is technically not similar to programming. We can't change how exclusive programs behave.

By the way you can find and open filenames when they have Uppercase letters and spaces. They are not a problem. The problem is that we use sed (because it's lighter and more efficient than other programs in this task) with regular expressions. So when your file names have things that can be interpreted as regular expression characters then you may not find or open them. But uppercase letters or spaces won't cause a problem. We can use awk or another approach here but it's an edge case and there is no need for that. Users can do the changes themselves if needed since this script is extremely minimal and therefore extensible. Maybe you mean the dmenu filtering should be case insensitive. Okay let's do it.

EDIT: I removed the error you see on the terminal. I made the dmenu filtering case insensitive. Have fun!

@TheYellowArchitect
Copy link
Contributor

TheYellowArchitect commented Nov 19, 2023

By the way, on Rofi, there is an option called -auto-select that selects the matching entry if it's the only one, automatically without enter like you mentioned for "@@". On Dmenu as far as I know, it's not possible but it can be made possible with patches. I can't do anything about it in the script. Shell scripting is technically not similar to programming. We can't change how exclusive programs behave.

I understand. I myself barely wrote for dmenu, so I didn't know if dmenu can do accomplish that.

Btw for merging it, I suggest just adding a comment at the top of the bin, saying basically:

#run `updatedb -o ~/.config/.mymlocate.db -U /VideosPath` to populate the database to be parsed below

This script can be made more robust in general but I prefer simplicity for jobs like this instead of extreme correctness: "Worse is Better".

I agree that for developers or specific users, simplicity is preferred instead of adding many error checks. It's how programs bloat after all (gotta cover 0.1% edge use-case)
Honestly, it's 3 lines so keep it as is, I was personally surprised when I read the wall of text expecting some huge script, then see 3 lines. Makes anyone happy to see it, because after all, computers were meant to be simple and easy to configure.

And thanks but no. I didn't just watch that video and write the script :). I am very interested in C and shell scripting even though I am not a programmer myself. I try to write extremely minimalist scripts by achieving the same targets. I generally prefer Dash and I use Bash only if I feel like I have to use it.

I am happy to hear it, and generally to see more and more people who aren't full-time programmers, use programming to aid their daily activities, whatever these are. Kinda like Luke
Btw with your knowledge in C and shell-scripting, I would argue you are a programmer, you just aren't paid for it lol

You can use my other script on another pull request named "fixnames" to make your file names Unix compatible and increase your consistency. It's extremely fast.

I didn't find this but I assume the below replaces it

EDIT: I removed the error you see on the terminal. I made the dmenu filtering case insensitive. Have fun!

Thank you very much. I have an external drive which contains ~1 TB of videos (I basically hoard all good videos, and also have one for "to-watch later"), this script is very helpful, it never passed my mind to make something like this, thanks :D

@emrakyz
Copy link
Contributor Author

emrakyz commented Nov 19, 2023

Here you can find the "fixnames" script "It's a very small script again with good explanations": #1322

I think you will like this too. These scripts can also be good for you if you try to understand shell scripting or improve your skills. There are interesting ideas in them.

For the comment "recommendation", I generally prefer robust documentation instead of adding it to the script as a comment but for this it can be added of course. I can also make it automatic but we need to make sure dmenu prompt works for you to put the second input.

@TheYellowArchitect
Copy link
Contributor

TheYellowArchitect commented Nov 19, 2023

Here you can find the "fixnames" script "It's a very small script again with good explanations": #1322

I would use it, except a) Remove non-English characters. would remove greek and japanese characters, so it would cause problems for me. Seems pretty good though.

These scripts can also be good for you if you try to understand shell scripting or improve your skills. There are interesting ideas in them.

Indeed. These days I am configuring this computer, and I had to learn the basics of bash (still not familiar with much stuff), the last 4~5 days have been great

The best part I think is when you open some random command in the same config file you are writing (or usually in .local/bin) and basically understand how it functions. This is true for all code ofc, you use what works and is often-used in the codebase, it's the fastest way to learn (trial&error and understanding why it gave error, repeat)

Related to the above is that the best documentation is the code itself, it should be so clear that it's self-explanatory. Ideally, the code would be so clean that the function names and variable names are descriptive and accurate of their functions (and good coding practices like a function having no more than 2 depths), and hence no comments or documentation is ever needed.

The above cannot be applied to bash however, as it was made in a different time, and it's goal is to be as small and performant as possible. chain operations instead of if/else, pipes instead of proper input parameters, and the infamous obfuscated variable names or esoteric symbols which have rules of their own.
For this reason, I suggest a simple comment, because I believe the user who will use this script, will browse these .local/bin scripts out of curiosity what he has available. He will stumble into this, read it, and run the command, job done. No need to auto-run for him or anything, these 3 lines are clean, no need to triple the codesize (there are users with systemd, runit, openrc, how would you create a service for it?)

As for robust documentation, documentation which is seperated from the code is bad imo, users never bother searching for it (also such documentation decays rapidly, but luckily not here since it's just 3 lines without intention of addition)

@emrakyz
Copy link
Contributor Author

emrakyz commented Nov 19, 2023

I would still recommend naming your files with English character equivalents instead of using Greek or Japanese characters. It makes things much more consistent and less problematic to deal with. I am not sure how easy it would be though for your use case. In Turkish for example, it's easy to replace "çatı" with "cati".

@emrakyz
Copy link
Contributor Author

emrakyz commented Nov 19, 2023

@TheYellowArchitect

Btw for merging it, I suggest just adding a comment at the top of the bin, saying basically:
#run updatedb -o ~/.config/.mymlocate.db -U /VideosPath to populate the database to be parsed below

I automated the process. If locate is not installed, the script will install them for you and give you related notifications.

If locate is installed but there is no database, then it will prompt you asking in dmenu for the path of the disk with your videos. You can enter "/mnt/harddisk" here for example. Then it will continue updating the database.

So we covered the error cases here and it's much more intuitive now. But it's 22 lines instead of 3 :) The efficiency of the script won't change though. The checks are pretty minimal and most of the code won't run if everything is okay.

@TheYellowArchitect
Copy link
Contributor

22 lines instead of 3 spooked me, but seeing the commited file, its still basically 3 lines, seperated by 2 clear condition codeblocks, so it's a good change. Though the locate part could be just 1 comment line which asks the user, and the database part uses rofi which should be included in the pacman locate codeblock. Those 2 codeblocks could be 2 comment lines instead, but no harm is done, the main part of the script (3 lines) works the same and I would feel bad to say the additions weren't required.

@emrakyz
Copy link
Contributor Author

emrakyz commented Nov 19, 2023

Yes I separated them to make it automatic and clear for its appearance and ease of change.

Yes. 22 lines are not valid for most users. The script basically only runs these checks. So if you pass these, the script is 3 lines for you :)

command -v locate >/dev/null
[ -s "$HOME/.config/.mymlocate.db" ]

@emrakyz
Copy link
Contributor Author

emrakyz commented Jun 3, 2024

@TheYellowArchitect You may want to check #1419

@emrakyz emrakyz closed this Jun 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants