模型描述 (Model Description)

StableSR是一种利用预训练文生图扩散模型中封装的先验知识进行盲超分的图像超分方法。该方法通过使用time-aware encoder，在不改变预训练文生图模型的情况下，实现高质量的图像修复，从而保留了预训练模型的生成效果，同时有效地减少训练成本。更多详情可参照主页

📃Paper |
📽️Video |

模型流程图

运行环境 (Operating environment)

Dependencies and Installation

Pytorch == 1.12.1
CUDA == 11.7
pytorch-lightning==1.4.2
xformers == 0.0.16 (Optional)
Other required packages in environment.yaml

# git clone the original repository
git clone https://github.com/IceClear/StableSR.git
cd StableSR

# Create a conda environment and activate it
conda env create --file environment.yaml
conda activate stablesr

# Install xformers
conda install xformers -c xformers/label/dev

# Install taming & clip
pip install -e git+https://github.com/CompVis/taming-transformers.git@master#egg=taming-transformers
pip install -e git+https://github.com/openai/CLIP.git@main#egg=clip
pip install -e .

代码范例 (Code example)

from modelscope.pipelines import pipeline
from modelscope.models import Model
from modelscope.outputs import OutputKeys
from PIL import Image
import os

input = "https://vigen-video.oss-cn-shanghai.aliyuncs.com/VideoEnhancement/Dataset/ClassicalSRDataset/inputs/OST_009.png"
model = Model.from_pretrained('xhlin129/cv_stablesr_image-super-resolution', steps=50)
inference = pipeline('stablesr-task', model=model)
output = inference(input, tile_overlap=16)
Image.fromarray(output[OutputKeys.OUTPUT_IMG]).save('result.png')
print(f'Output written to {os.path.abspath("result.png")}')

Citation

If you find our work helpful for your research, please consider citing the following BibTeX entry.

@inproceedings{wang2023exploiting,
    author = {Wang, Jianyi and Yue, Zongsheng and Zhou, Shangchen and Chan, Kelvin CK and Loy, Chen Change},
    title = {Exploiting Diffusion Prior for Real-World Image Super-Resolution},
    booktitle = {arXiv preprint arXiv:2305.07015},
    year = {2023}
}