Frequently used Linux Command Collections

By MasterData Science, Reference

Linux Basic Command

Linux Version Check

cat /etc/issue

Check File Exists

find | grep file_path

CPU Information

cat /proc/cpuinfo

move directory, file

mv original_path move_path

remove directory

rm -rf path

rename directory

mv /home/user/oldname /home/user/newname

zip directory (zip, tar.xz)

zip -r name.zip target_files
tar -Jcvf name.tar.xz target_files
(simple) tar cvf name.tar.xz target_files

unzip directory (zip, tar.xz)

unzip name.zip -d directory_name
tar -Jxvf name.tar.xz -C unzip_path
(simple) tar xvf name.tar.xz

check capacity

du -sh
(remained capacity) df -h
(for server command) ssh MASTER '/usr/sbin/xfs_quota -x -c "report -h -u" /home' 2> /dev/null

count directory number

ls -l | grep ^d | wc -l

count file number

ls -l | grep ^- | wc -l

make command

vi ~/.bashrc
alias command='original_command'
source ~/.bashrc

scp command

scp file_path id@ip:path_to_reveive
(i.e) scp file.zip anonymous@111.111.111.11:/share0/sha


VIM Command

go start line

gg

go end line

G

select all

shift+v+g

delete & copy

dd (line delete)
d (all delete)
y (all copy)


Command for coding

Python path

which python

check tensorboard

tensorboard --logdir=path

check gpu

watch -n 1 nvidia-smi

check cpu & ram

htop

pip update

pip install --upgrade pip

Register Path in Python File

import sys
sys.path.append('.')

python interpreter path check in code

import sys
sys.excutable

Appropriate PyTorch Version for me

# version.1
pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html

# version.2
pip install torch==1.11.0+cu113 torchvision==0.12.0+cu113 --extra-index-url https://download.pytorch.org/whl/cu113

PyTorch version and CUDA capability check

import torch
torch.cuda.get_device_name(0)
torch.cuda.is_available()
torch.__version__

(Anaconda3) enroll conda path
– Installing on Linux : https://docs.anaconda.com/anaconda/install/linux/

export PATH=~/anaconda3/bin:$PATH
conda --version

(Anaconda3) Create and activate a conda environment

conda create -n env_name python=3.8
conda activate env_name

(error) -> source ~/anaconda3/etc/profile.d/conda.sh

= enroll error code =
vi ~/.bashrc -> [copy/paste] -> [save] -> source ~/.bashrc

(Anaconda3) Environment List

conda info --envs

(Anaconda3) Remove Virtual Experiment

conda remove --name env_name --all

(Anaconda3) Install package

conda install -c package_name package_name=version

(Anaconda3) Conda Initialization

<bashrc>
conda deactivate
conda activate transt
export PYTHONPATH="(python path)":$PYTHONPATH


GPU Access Command
(AI Data Center, Yonsei University)

1. srun (max:6:00:00)

# register gpu
srun  --gres=gpu:1  --time=1:00:00  --pty bash -i

# check gpu
squeue -u $USER

# return gpu
scancel [job id]

2. sbatch (max:72:00:00)

# register gpu
sbatch --gres=gpu:1 --time=72:00:00 file_name.sh
sbatch -q big_qos --gres=gpu:1 --time=72:00:00 file_name.sh
sbatch -p suma_rtx4090 --gres=gpu:1 --time=72:00:00 file_name.sh
sbatch -p suma_rtx4090 -q big_qos --gres=gpu:1 --time=72:00:00 file_name.sh

# check gpu
squeue -u $USER

# check work start time
squeue -u $USER --start

# return gpu
scancel [job id]

ex) file_name.sh

#!/bin/bash
#SBATCH -J ahnsunghyun_resnet_test
#SBATCH -o resnet_test.txt
echo "### START DATE=$(date)"
python -u ltr/run_training.py transt transt
echo "### END DATE=$(date)"

(TIP) sbatch with jupyter

1. You can access Jupiter Lab by entering the sbatch command using the sh file written below.
2. To use sbatch with jupyter in VSCode, you must press (ctrl+’) and then enter the PORTS tab to set up Port and Local Address.
ex) Port: node14:8888, Local Address: localhost:8888
3. For Jupiter access address, check ‘jupyter.log’ file!
4. You should create an ipynb file, select the kernel, and type url in the same format (http://node02:8888/lab?token=…).
5. Choose virtual environment you entered or ‘Python 3 (ipykernel)’

#!/bin/bash
#SBATCH --job-name=jupyter
#SBATCH --output=./jupyter.log

jupyter lab --ip=0.0.0.0 --port=8888


VSCode SSH Command

show all command: <ctrl>+<shift>+<p>
ssh config path: C:\Users\user\.ssh\config

# Read more about SSH config files: https://linux.die.net/man/5/ssh_config
Host ahnsunghyun@127.0.0.1
    HostName 127.0.0.1
        User ahnsunghyun
        Port 22


Jupyter Connect on VSCode

1. ctrl+shift+p
2. specify local or remote jupyter server for connections
3. Jupyter url copy & paste
– jupyter import error: https://bioinfoblog.tistory.com/21