Functionalities
1. Model Profiling
Basic profiling
- Kindle model provides profiling option.
- Please refer to API reference page for the detailed information.
from kindle import Model
model = Model("model.yaml")
profiler = model.profile(n_run=100, batch_size=32, input_size=(224, 224), verbose=True)
- Above code will print the profiling result as following.
Profiling result by 100 times running. Sorted by running order.
------------------------------------------------------------------------------------------------------
idx | Name | Time(Mean) | Time(Std) | Time(Total) | Rank | Ratio | Params |
------------------------------------------------------------------------------------------------------
0 | Conv | 1.40 ms | 566.86 μs | 139.69 ms | 1 | 36.08% | 616 |
1 | MaxPool | 410.78 μs | 136.45 μs | 41.08 ms | 3 | 10.61% | 0 |
2 | nn.Conv2d | 980.33 μs | 459.86 μs | 98.03 ms | 2 | 25.32% | 3,200 |
3 | nn.BatchNorm2d | 277.48 μs | 141.98 μs | 27.75 ms | 5 | 7.17% | 32 |
4 | nn.ReLU | 109.23 μs | 43.48 μs | 10.92 ms | 7 | 2.82% | 0 |
5 | MaxPool | 242.73 μs | 80.28 μs | 24.27 ms | 6 | 6.27% | 0 |
6 | Flatten | 18.18 μs | 16.98 μs | 1.82 ms | 10 | 0.47% | 0 |
7 | Linear | 294.72 μs | 192.97 μs | 29.47 ms | 4 | 7.61% | 123,000 |
8 | Linear | 94.48 μs | 44.85 μs | 9.45 ms | 8 | 2.44% | 10,164 |
9 | Linear | 46.44 μs | 26.23 μs | 4.64 ms | 9 | 1.20% | 850 |
------------------------------------------------------------------------------------------------------
Running time
- Total : 387.13 ms
- Mean : 3.87 ms
- STD : 1.45 ms
Profiling by time consumption order
- Profiling result can be sorted by running times.
profiler.print_result(sort_by_rank=True)
Profiling result by 100 times running. Sorted by time consumption.
------------------------------------------------------------------------------------------------------
idx | Name | Time(Mean) | Time(Std) | Time(Total) | Rank | Ratio | Params |
------------------------------------------------------------------------------------------------------
0 | Conv | 1.40 ms | 566.86 μs | 139.69 ms | 1 | 36.08% | 616 |
2 | nn.Conv2d | 980.33 μs | 459.86 μs | 98.03 ms | 2 | 25.32% | 3,200 |
1 | MaxPool | 410.78 μs | 136.45 μs | 41.08 ms | 3 | 10.61% | 0 |
7 | Linear | 294.72 μs | 192.97 μs | 29.47 ms | 4 | 7.61% | 123,000 |
3 | nn.BatchNorm2d | 277.48 μs | 141.98 μs | 27.75 ms | 5 | 7.17% | 32 |
5 | MaxPool | 242.73 μs | 80.28 μs | 24.27 ms | 6 | 6.27% | 0 |
4 | nn.ReLU | 109.23 μs | 43.48 μs | 10.92 ms | 7 | 2.82% | 0 |
8 | Linear | 94.48 μs | 44.85 μs | 9.45 ms | 8 | 2.44% | 10,164 |
9 | Linear | 46.44 μs | 26.23 μs | 4.64 ms | 9 | 1.20% | 850 |
6 | Flatten | 18.18 μs | 16.98 μs | 1.82 ms | 10 | 0.47% | 0 |
------------------------------------------------------------------------------------------------------
Running time
- Total : 387.13 ms
- Mean : 3.87 ms
- STD : 1.45 ms
2. Get MACs
- MACs is Multiply-accumulate operation which represents computational expense.
- Approximately 1 MACs = 0.5 * FLOPs
- Kindle is using ptflops for computing MACs.
mac = profiler.get_macs(verbose=True)
print(f"{mac:,0d} MACs")
Model(
0.138 M, 100.000% Params, 0.002 GMac, 100.000% MACs,
(model): Sequential(
0.138 M, 100.000% Params, 0.002 GMac, 100.000% MACs,
(0): Conv(
0.001 M, 0.447% Params, 0.001 GMac, 39.517% MACs,
(conv): Conv2d(0.001 M, 0.435% Params, 0.001 GMac, 37.997% MACs, 3, 8, kernel_size=(5, 5), stride=(1, 1), padding=[2], bias=False)
(batch_norm): BatchNorm2d(0.0 M, 0.012% Params, 0.0 GMac, 1.013% MACs, 8, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(activation): LeakyReLU(0.0 M, 0.000% Params, 0.0 GMac, 0.507% MACs, negative_slope=0.01)
)
(1): MaxPool2d(0.0 M, 0.000% Params, 0.0 GMac, 0.507% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(2): Conv2d(0.003 M, 2.321% Params, 0.001 GMac, 50.663% MACs, 8, 16, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
(3): BatchNorm2d(0.0 M, 0.023% Params, 0.0 GMac, 0.507% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(4): ReLU(0.0 M, 0.000% Params, 0.0 GMac, 0.253% MACs, inplace=True)
(5): MaxPool2d(0.0 M, 0.000% Params, 0.0 GMac, 0.253% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(6): Flatten(0.0 M, 0.000% Params, 0.0 GMac, 0.000% MACs, start_dim=1, end_dim=-1)
(7): Linear(
0.123 M, 89.220% Params, 0.0 GMac, 7.614% MACs,
(linear): Linear(0.123 M, 89.220% Params, 0.0 GMac, 7.607% MACs, in_features=1024, out_features=120, bias=True)
(activation): ReLU(0.0 M, 0.000% Params, 0.0 GMac, 0.007% MACs, )
)
(8): Linear(
0.01 M, 7.373% Params, 0.0 GMac, 0.634% MACs,
(linear): Linear(0.01 M, 7.373% Params, 0.0 GMac, 0.629% MACs, in_features=120, out_features=84, bias=True)
(activation): ReLU(0.0 M, 0.000% Params, 0.0 GMac, 0.005% MACs, )
)
(9): Linear(
0.001 M, 0.617% Params, 0.0 GMac, 0.053% MACs,
(linear): Linear(0.001 M, 0.617% Params, 0.0 GMac, 0.053% MACs, in_features=84, out_features=10, bias=True)
(activation): Identity(0.0 M, 0.000% Params, 0.0 GMac, 0.000% MACs, )
)
)
)
1,616,970 MACs
3. Test Time Augmentation
- Kindle model supports TTA with simple usage.
- Reference document: https://limjk.ai/kindle/api/kindle.model/#kindle.model.Model.forward
TTA with explicit functions
import torch
import torch.nn.functional as F
import numpy as np
from kindle import Model
@torch.no_grad()
def aug_flip_lr(x: torch.Tensor) -> torch.Tensor:
return torch.flip(x, dims=[-1])
@torch.no_grad()
def aug_flip_ud(x: torch.Tensor) -> torch.Tensor:
return torch.flip(x, dims=[-2])
@torch.no_grad()
def aug_random_scale(x: torch.Tensor) -> torch.Tensor:
ratio = np.random.uniform(0.8, 1.2)
xr = F.interpolate(x, scale_factor=ratio)
if xr.shape[2:] != x.shape[2:]:
h1, w1 = x.shape[2:]
h2, w2 = xr.shape[2:]
xr = F.pad(xr, [0, w1 - w2, 0, h1 - h2], value=0)
return xr
aug_funcs = [aug_flip_lr, aug_flip_ud, aug_random_scale]
model = Model("model.yaml")
model_in = torch.rand(1, 3, 32, 32)
model_out = model(model_in, augment_func=aug_funcs)
model_out
in the above code contains 4 elements which are outputs of the model with each 3 augmentations and originalmodel_in
.
TTA with repeating one augmentation function
- If a single augmentation function is given to
augment_func
, TTA will consider as random augmentation and run the network repeatingn_augment
times.
model_out = model(model_in, augment_func=aug_random_scale, n_augment = 5)
model_out
in the above code contains 6 elements which are outputs of the model with 5aug_random_scale
applied and originalmodel_in
.