Progmar Marcin Załęczny

Language:
We are using cookies in the page. If you use the page you agree for the cookies.      Close

How to make Ubuntu repository

First we make sure that we have php5 command line interpreter installed: sudo apt-get install php5-cli

Next we obtain list of all deb packages in Ubuntu repository. apt-cache pkgnames > pkg_list.txt We get the list in pkg_list.txt file.

When we have packages list we can start downloading them from Ubuntu server. We do it by using following php5 script ubuntu-repo-download.php:


<?php

$lines 
file('pkg_list.txt');

foreach (
$lines as $pkg) {
    
$pkg trim($pkg);
    
$cmd "apt-get download $pkg";
    echo 
$cmd "\n";
    
system($cmd);
}

The script downloads all listed packages to a current directory. We call the script by typing in command line: php5 ubuntu-repo-download.php

When large amount of packages is downloaded we can stop download process by pressing Ctrl+C. Now we can divide packages into the one letter subdirectories. Script ubuntu-repo-split-files-into-subdirectories.php helps us with the task:


<?php

$srcdir 
"/mnt/sdb1/repository";
$destdir "/mnt/sdb1/repository";

if (
is_dir($srcdir)) {
    
$files scandir($srcdir);
    foreach (
$files as $file) {
        if (
is_file("$srcdir/$file")) {
            
$letter $file{0};
            if (!
is_dir("$destdir/$letter")) {
                
mkdir("$destdir/$letter");
            }
            if (
is_dir("$destdir/$letter")) {
                echo 
"Moving file: $file\n";
                
rename("$srcdir/$file""$destdir/$letter/$file");
            }
        }
    }
}

Before we run the script we have to set proper values to variables $srcdir and $destdir. Both of the variables shoud be set to the downloaded packages' directory. We run the script with following command: php5 ubuntu-repo-split-files-into-subdirectories.php

Before continueing download process, we have to update packages list. This task is resolved by following script ubuntu-repo-check-if-downloaded.php:


<?php

$repo_dir 
"/mnt/sdb1/repository";

$lines file('pkg_list.txt');
$content "";

$counter 0;

foreach (
$lines as $pkg) {
    
$pkg trim($pkg);
    
$letter $pkg{0};
    
$cmd "ls " $repo_dir "/" $letter "/" $pkg "_* 2>/dev/null";
    
//echo $cmd . "\n";
    
$result system($cmd);
    if (!empty(
$result)) {
        echo 
$pkg." is already downloaded\n";
        
$counter += 1;
    } else {
        
$content .= $pkg "\n";
    }
}

file_put_contents('pkg_list.txt'$content);

The script works on subdirectory structure generated by previous script (ubuntu-repo-split-files-into-subdirectories.php). Before calling the script we have to set proper value to $repo_dir variable. We call it as follows: php5 ubuntu-repo-check-if-downloaded.php

Sometimes it happens that downloaded packages are files of 0 size. We maust remove them and download again. We do it by using script ubuntu-repo-remove-empty-files.php:


<?php

$dir 
"/mnt/sdb1/repository";

$files scandir($dir);
foreach (
$files as $file) {
    if (
is_file("$dir/$file")) {
        if (
filesize("$dir/$file") == 0) {
            echo 
"Removing file: $dir/$file\n";
            
unlink("$dir/$file");
        }
    }
    if (
is_dir("$dir/$file")) {
        
$pkgs scandir("$dir/$file");
        foreach (
$pkgs as $pkg) {
            if (
is_file("$dir/$file/$pkg")) {
                if (
filesize("$dir/$file/$pkg") == 0) {
                       echo 
"Removing file: $dir/$file/$pkg\n";
                    
unlink("$dir/$file/$pkg");
                }
            }
        }
    }
}

Before we run the script we must set proper value to $repo_dir variable. Next we run the script: php5 ubuntu-repo-remove-empty-files.php After removing empty files we must refresh packages for download list and run downloading process.

When all files are downloaded we make repository of them. We create directory /var/www/repository and inside it subdirectories /var/www/repository/repo_01 .. /var/www/repository/repo_09. Each of subdirectories repo_0x should contain symlinks to three one letter directories containing packages. For example /var/www/repository/repo_01 directory should contain symlinks to a, b, c directories. /var/www/repository/repo_02 directory should contain symlinks to d, e, f directories, and so on.

Now we enter each of /var/www/repository/repo_01 .. /var/www/repository/repo_09 directories and run following command inside them: dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz Be prepared that this is very time consuming process.

When we have all Packages.gz files, we configure client machine to get packages from our web server. Add following lines to the /etc/apt/sources.list file (near the top) on client machine: deb http://localhost/repository/repo_01 /repo_01
deb http://localhost/repository/repo_02 /repo_02
...
deb http://localhost/repository/repo_02 /repo_09