|
16 | 16 |
|
17 | 17 | This tutorial will walk you through the basics of using the usb lighting package.
|
18 | 18 | Let's get started by training a FreeMatch/SoftMatch model on CIFAR-10 using pre-trained ViT!
|
19 |
| -And we will show it is easy to change the semi-supervised algorthm and train on imbalanced datasets. |
| 19 | +And we will show it is easy to change the semi-supervised algorithm and train on imbalanced datasets. |
20 | 20 |
|
21 | 21 |
|
22 | 22 | .. figure:: /_static/img/usb_semisup_learn/code.png
|
|
29 | 29 | # --------------------
|
30 | 30 | # Here we provide a brief introduction to FreeMatch and SoftMatch.
|
31 | 31 | # First we introduce a famous baseline for semi-supervised learning called FixMatch.
|
32 |
| -# FixMatch is a very simple framework for semi-supervised learning, where it utlizes a strong augmentation to generate pseudo labels for unlabeled data. |
| 32 | +# FixMatch is a very simple framework for semi-supervised learning, where it utilizes a strong augmentation to generate pseudo labels for unlabeled data. |
33 | 33 | # It adopts a confidence thresholding strategy to filter out the low-confidence pseudo labels with a fixed threshold set.
|
34 | 34 | # FreeMatch and SoftMatch are two algorithms that improve upon FixMatch.
|
35 | 35 | # FreeMatch proposes adaptive thresholding strategy to replace the fixed thresholding strategy in FixMatch.
|
36 | 36 | # The adaptive thresholding progressively increases the threshold according to the learning status of the model on each class.
|
37 | 37 | # SoftMatch absorbs the idea of confidence thresholding as an weighting mechanism.
|
38 | 38 | # It proposes a Gaussian weighting mechanism to overcome the quantity-quality trade-off in pseudo-labels.
|
39 |
| -# In this toturial, we will use USB to train FreeMatch and SoftMatch. |
| 39 | +# In this tutorial, we will use USB to train FreeMatch and SoftMatch. |
40 | 40 |
|
41 | 41 |
|
42 | 42 | ######################################################################
|
|
46 | 46 | # It is easy-to-use/extend, affordable to small groups, and comprehensive for developing and evaluating SSL algorithms.
|
47 | 47 | # USB provides the implementation of 14 SSL algorithms based on Consistency Regularization, and 15 tasks for evaluation from CV, NLP, and Audio domain.
|
48 | 48 | # It has a modular design that allows users to easily extend the package by adding new algorithms and tasks.
|
49 |
| -# It also supprts a python api for easier adaptation to different SSL algorithms on new data. |
| 49 | +# It also supports a python api for easier adaptation to different SSL algorithms on new data. |
50 | 50 | #
|
51 | 51 | #
|
52 | 52 | # Now, let's use USB to train FreeMatch and SoftMatch on CIFAR-10.
|
|
57 | 57 | # - ``get_net_builder`` to create a model, here we use pre-trained ViT
|
58 | 58 | # - ``get_algorithm`` to create the semi-supervised learning algorithm, here we use FreeMatch and SoftMatch
|
59 | 59 | # - ``get_config``: to get default configuration of the algorithm
|
60 |
| -# - ``Trainer``: a Traner class for training and evaluating the algorithm on dataset |
| 60 | +# - ``Trainer``: a Trainer class for training and evaluating the algorithm on dataset |
61 | 61 | #
|
62 | 62 | import semilearn
|
63 | 63 | from semilearn import get_dataset, get_data_loader, get_net_builder, get_algorithm, get_config, Trainer
|
|
130 | 130 |
|
131 | 131 | ######################################################################
|
132 | 132 | # Finally, let's evaluate the trained model on validation set.
|
133 |
| -# After training 4000 itertaions with FreeMatch on only 40 labels of CIFAR-10, we obtain a classifier that achieves above 93 accuracy on validation set. |
| 133 | +# After training 4000 iterations with FreeMatch on only 40 labels of CIFAR-10, we obtain a classifier that achieves above 93 accuracy on validation set. |
134 | 134 | trainer.evaluate(eval_loader)
|
135 | 135 |
|
136 | 136 |
|
|
140 | 140 | # --------------------
|
141 | 141 | #
|
142 | 142 | # Now let's say we have imbalanced labeled set and unlabeled set of CIFAR-10, and we want to train a SoftMatch model on it.
|
143 |
| -# We create an imbalanced labeld set and imbalanced unlabelde set of CIFAR-10, by setting the ``lb_imb_ratio`` and ``ulb_imb_ratio`` to 10. |
| 143 | +# We create an imbalanced labeled set and imbalanced unlabeled set of CIFAR-10, by setting the ``lb_imb_ratio`` and ``ulb_imb_ratio`` to 10. |
144 | 144 | # Also we replace the ``algorithm`` with ``softmatch`` and set the ``imbalanced`` to ``True``.
|
145 | 145 | #
|
146 | 146 | config = {
|
|
0 commit comments