Let's Run Jinyeah

[Window] Python 설정

jinyeah — Mon, 24 Oct 2022 15:03:04 +0900

python 위치

where python
>> C:\Users\samsung\Anaconda3\python.exe

[Anaconda3] python 가상환경

위치: C:\Users\samsung\Anaconda3\envs

conda envs

[Anacodna3] python 버전 확인

python --version
>>Python 3.6.8 :: Anaconda, Inc.

[Anaconda3] 설치 package 확인

conda list

Outer product, Inner product

jinyeah — Thu, 6 Oct 2022 05:02:44 +0900

Outer product

uvT
output: matrix
time complexity: O(n2)

Inner product

uTv
output: scalar
time complexity: O(n)
- a = [a0, a1, a2, ... , aN-1], b = [b0, b1, b2, ...., bN-1]
- a0*b0 + a1*b1 + aN-1*bN-1
- Assuming that multiplication and addition are constant-time operations, the time-complexity is O(n)
  - Multiply O(n) + add O(n) = O(n)

Top: Inner product, Down: Outer product

[Shadowing] 220925

jinyeah — Mon, 26 Sep 2022 06:45:59 +0900

The Ellen Show "The Unbelievably Hilarious Amy Schumer"

1. I don't fit in here -- just straight up body type

Straight up - 격한공감, 완전, absolutely, totally / 솔직하게, honestly

ex) I'm telling you straight up, I never saw him with her.

ex) Straight up, I didn't think you'd make it this far in the competition, but you've done really well.

2. my arms register as legs

- to put information(your name) into an official list or record

ex) Students have to register for the new course by the end of April.

- to show or express something

ex) His face registered extreme disapproval of what he had witnessed.

3. bawling

bawl - to shout in a very loud vice

4. courtside tickets to a Laker game

courtside in basketball - the area around the actual court

5. my business agent thought I was mad at him for sexually harassing me

harass - to continue to annoy or upset someone over a period of time

6. I thought it would be free booze

free booze - a version of an alcoholic drink made without alcohol, or with the alcohol removed or reduced to almost zero

would - will보다 약한 확신

7. To pull off that name

pull off sth - to succeed in doing sth difficult or unexpected

putt sth off - to manage to do sth difficult

8. Like if my name were "Quinn" on a show, they'd be like "Oh, the jolly Irish groundskeeper, that~"

groundskeeper - 관리인

If S were N, S would be like ~

9. My resting face is just a scowl

scowl - a very annoyed expression

10. I have what I am calling "at-risk chin"

chin - the part of a person's face below their mouth(턱)

11. I'm pounding red wine

pound - to hit repeatedly with force, or to crush by hitting repeatedly

ex) The speaker pounded his fists on the table

12. I look like her if she were stung by a million bees

sting - (벌레가) 물다

sting stang stung

13. I want a trough and I want to dunk my head in it

trough - a long, narrow container without a lid that usually holds water or food for farm animals

duck - to put something into liquid for a short time

14. you can't, and not just because I'm not totally out of the woods with this UTI

out of the woods - no longer be in danger of difficulty

UTI - an infection in any part of the urinary system

15. check it out

to examine or try something

16. I went to the scone and I kind of just sidled up to it

sidle up - to walk toward or away from someone, trying not to be noticed

표현

1. I get red wine teeth right out the gate

2. I was eyeing this particular scone.

Transfer Learning and Domain Adaptation

jinyeah — Wed, 10 Aug 2022 20:43:38 +0900

“Domain” and “Task”

Domain relates to the feature space of a specific dataset and the marginal probability distribution of features
Task relates to the label space of a dataset and an objective predictive function

Transfer Learning

goal is to transfer the knowledge learned from the Task(a) on Domain A to the Task(b) on Domain B
common to update the last several layers of the pre-trained network towards a new given task with a new label space (the supporting assumption is that the early layers in the network extract low-level features which are universal for vision tasks)
- Target Domain에 label이 없다면 학습이 불가능

Domain Adaptation

particular type of transfer learning
goal is to transfer knowledge from S Domain to T Domain to perform a specific Task on T (Domain Shift)
- Task is shared
- the marginal distributions are different between the source(S) Domain and target(T) Domain

Types of Domain Adaptation

Supervised DA
- small number labeled data in the target domain are available
- pre-train the model with source dataset and fine-tune with target dataset (Transfer learning)
Semi-supervised DA
- small number of labeled data as well as redundant unlabeled in the target domain are available
Unsupervised DA / Self-supervised DA
- only unlabeled target data are available
- 대표논문. [DANN] Domain-Adversarial Training of Neural Networks
- self-supervised a branch of unsupervised learning: Obtain labels form the data itself
Source-free DA
- only a source pre-trained model and an unlabeled target domain dataset are available
- 대표논문. [DistillAdapt] Source-Free Activate Visual Domain Adaptation

Reference

https://lhw0772.medium.com/study-da-domain-adaptation-%EC%95%8C%EC%95%84%EB%B3%B4%EA%B8%B0-%EA%B8%B0%EB%B3%B8%ED%8E%B8-4af4ab63f871

Guan, Hao, and Mingxia Liu. "Domain adaptation for medical image analysis: a survey." IEEE Transactions on Biomedical Engineering 69.3 (2021)

Linux Disk 확인 및 Filesystem 용량 확인

jinyeah — Mon, 8 Aug 2022 17:07:46 +0900

Display all of your drives on a Linux System

sudo fdisk -l
lsblk

총 2개의 disk (nvme0n1과 nvme1n1)가 마운트 되어 있음
nvme0n1의 마운트 위치: /
nvme1n1의 마운트 위치: /data1

Display Size of all of your drives on a Linux System

df -h

특정 디렉토리의 File system 용량 확인하기

du -sh # 총 용량
du -h # 모든 하위 디렉토리들의 용량
du -h --max-depth=1 # 첫번째 하위 디렉토리들의 용량

Entropy and Cross-Entropy

jinyeah — Sun, 31 Jul 2022 00:05:24 +0900

Entropy는 일반적으로 불확실성을 나타내는 지표이다. 딥러닝에서는 이를 정보량으로 볼 수 있다.

Rationale

Information ( = i) = -㏒ ( = i)

The degree of information delivered by an event xi
- is low if ( = i) is close to 1
- is high if ( = i) is close to 0
즉, 확률이 클수록 정보량이 적다
- logarithm: -logx = x가 1에 가까울수록 작다

Expectation

E [ ] = ∑ · ( = )

Summation of (특정 outcome x 그 outcome이 나올 확률)

Entropy

H [ ] = E [Information( = )] = - ∑ ( = ) · ㏒ ( = )

(특정 값에 대한 정보량)의 기댓값
Expected value of the uncertainty of its outcomes

Cross-Entropy

H [Xt, Xp] = E [Information( = )] = - ∑ t( t= ) · ㏒ p( p= )

t: true probability distribution

p: approximate probability distribution

정답 분포와 예측 분포의 outcome이 동일하고 정답 분포를 알 경우 활용
minimize when t≈ p and is large
- 둘 다 높은 확률이면 0에 수렴하여 cross-entropy loss가 작아짐
- 둘 다 낮은 확률이면 무한대에 수렵하여 cross-entropy loss 커짐
- 하나는 높고, 하나는 낮은 확률이면( t≉ p), 한 값이 크기 때문에 cross-entropy loss 커짐

[Python] MRI Resampling

jinyeah — Fri, 8 Jul 2022 17:33:26 +0900

When to Resample?

Anytime we use two datasets with different sized voxels

Dicom Attributes to use

1. Slice Thickess

2. PixelSpacing (width, height)

Calcuate new size

out_size = [
        int(np.round(original_size[0] * (original_spacing[0] / out_spacing[0]))),
        int(np.round(original_size[1] * (original_spacing[1] / out_spacing[1]))),
        int(np.round(original_size[2] * (original_spacing[2] / out_spacing[2])))]

How to resample with SimpleITK

def resample_img(itk_image, out_spacing=[2.0, 2.0, 2.0], is_label=False):
    # resample images to 2mm spacing with simple itk

    original_spacing = itk_image.GetSpacing()
    original_size = itk_image.GetSize()

    out_size = [
        int(np.round(original_size[0] * (original_spacing[0] / out_spacing[0]))),
        int(np.round(original_size[1] * (original_spacing[1] / out_spacing[1]))),
        int(np.round(original_size[2] * (original_spacing[2] / out_spacing[2])))]

    resample = sitk.ResampleImageFilter()
    resample.SetOutputSpacing(out_spacing)
    resample.SetSize(out_size)
    resample.SetOutputDirection(itk_image.GetDirection())
    resample.SetOutputOrigin(itk_image.GetOrigin())
    resample.SetTransform(sitk.Transform())
    resample.SetDefaultPixelValue(itk_image.GetPixelIDValue())

    if is_label:
        resample.SetInterpolator(sitk.sitkNearestNeighbor)
    else:
        resample.SetInterpolator(sitk.sitkBSpline)

    return resample.Execute(itk_image)

How to resample with Python - Skimage resize

def interpolate(img, new_size):
    new_img = resize(img, new_size, order=1, anti_aliasing=False, clip=False, preserve_range=True)
    return new_img

reference

https://f-i-tushar-eee.medium.com/3d-medical-imaging-pre-processing-all-you-need-6ba981738877

Normalization

jinyeah — Sat, 18 Jun 2022 17:31:36 +0900

What is the normalization formula used for?

Normalization is useful in statistics for creating a common scale to compare data sets with very different values.

Deep Learning view?

학습의 안정화: Gradient vanising/exploding 문제를 해결할 수 있음
학습시간의 단축: learning rate를 크게 할 수 있음
성능 개선: local optimum에서 빨리 빠져나올 수 있음

Min-Max Normalization

Method

normalization formula to [0,1] xnormalized = (x-xmin) / (xmax-xmin)

if x==xmin, xnormalized = 0
if x==xmax, xnormalized = 1

Normalization formula for custom ranges(a,b) xnormalized = a + (((x-xmin)*(b-a)) / (xmax-xmin))

if x==xmin, xnormalized = a
if x==xmax, xnormalized = a+(b-a) = b

Summary

guaranteed to reshape both of our features to be between 0 and 1
MinMax Scaling is that it is highly influenced by the maximum and minimum values in our data so if our data contains outliers it is going to be biased
might compress all inliers in a narrow range if affected by outliers
- Normalizing fixed the squishing problem on the y-axis, but the x-axis is still problematic. Now if we were to compare these points, the y-axis would dominate; the y-axis can differ by 1, but the x-axis can only differ by 0.4.

Standardization

Method

scales the features so that they have μ=0 and σ=1

μ : mean value of the feature
σ : standard deviation of the feature

Z-Score Normalization

normalization formula xnormalized = (x-μ)/σ

if x==mean of the all the values of the feature, xnormalized = 0
if x < mean of the all the values of the feature, xnormalized < 0

Summary

avoids this outlier issue
The size of those negative, positive numbers is determined by the standard deviation of the feature. (if large standard deviation, the normalized values will be closer to 0)

Pytorch Normalization (Standardization)

각 channel별로 mean, std 값 할당

transform = torchvision.transforms.Compose([
    torchvision.transforms.ToTensor(),
    torchvision.transforms.Normalize(
        mean=[mean_1, mean_2, mean_3],
        std=[std_1, std_2, std_3],
    ),
])

reference

https://www.codecademy.com/article/normalization

https://www.indeed.com/career-advice/career-development/normalization-formula

Timeline

jinyeah — Sat, 18 Jun 2022 15:47:35 +0900

2022 Fall 미국 컴퓨터공학 석사 지원

COVID-19으로 GRE 점수 없이 지원

막학기(2021-2학기)에 1학점을 들으면서 대학원 준비

Timeline

8, 9월: TOFEL 공부 (8월부터 공부시작했고 원하는 점수를 얻은 건 11월)

9월: 교수님 면담, 추천서 요청

10월: CV작성, SOP 개요 잡기

11월, 12월: SOP, PS 작성

12월, 1월, 2월: 지원

3월, 4월, 5월: 결과 발표

현재

3월에 합격소식 받고 4월부터 국내 대학원 연구실에서 연구중(8월까지)

Numpy 모음

jinyeah — Fri, 17 Jun 2022 03:07:49 +0900

Statistic methods

np.sort

np.min() & np.max() & np.median()

np.percentile

calculate the ith percentile of the input numpy array along a specified axis

ith percentile is the value at which i percent of the data is below it
axis (default): input array is flattened

np.histogram

Find index or value

np.where

Change the shape

np.squeeze

remove 1-dimensional axis

np.tile

transpose & reshape

ravel & flatten

Generate an array

np.zeros & np.zeros_like

np.ones & np.ones_like

np.random

np.arange

Load & Save

np.load & np.save