Progmar Marcin Załęczny

Language:

How to make Ubuntu repository

First we make sure that we have php5 command line interpreter installed: sudo apt-get install php5-cli

Next we obtain list of all deb packages in Ubuntu repository. apt-cache pkgnames > pkg_list.txt We get the list in pkg_list.txt file.

When we have packages list we can start downloading them from Ubuntu server. We do it by using following php5 script ubuntu-repo-download.php:


<?php

$lines 
file('pkg_list.txt');

foreach (
$lines as $pkg) {
    
$pkg trim($pkg);
    
$cmd "apt-get download $pkg";
    echo 
$cmd "\n";
    
system($cmd);
}

The script downloads all listed packages to a current directory. We call the script by typing in command line: php5 ubuntu-repo-download.php

When large amount of packages is downloaded we can stop download process by pressing Ctrl+C. Now we can divide packages into the one letter subdirectories. Script ubuntu-repo-split-files-into-subdirectories.php helps us with the task:


<?php

$srcdir 
"/mnt/sdb1/repository";
$destdir "/mnt/sdb1/repository";

if (
is_dir($srcdir)) {
    
$files scandir($srcdir);
    foreach (
$files as $file) {
        if (
is_file("$srcdir/$file")) {
            
$letter $file{0};
            if (!
is_dir("$destdir/$letter")) {
                
mkdir("$destdir/$letter");
            }
            if (
is_dir("$destdir/$letter")) {
                echo 
"Moving file: $file\n";
                
rename("$srcdir/$file""$destdir/$letter/$file");
            }
        }
    }
}

Before we run the script we have to set proper values to variables $srcdir and $destdir. Both of the variables shoud be set to the downloaded packages' directory. We run the script with following command: php5 ubuntu-repo-split-files-into-subdirectories.php

Before continueing download process, we have to update packages list. This task is resolved by following script ubuntu-repo-check-if-downloaded.php:


<?php

$repo_dir 
"/mnt/sdb1/repository";

$lines file('pkg_list.txt');
$content "";

$counter 0;

foreach (
$lines as $pkg) {
    
$pkg trim($pkg);
    
$letter $pkg{0};
    
$cmd "ls " $repo_dir "/" $letter "/" $pkg "_* 2>/dev/null";
    
//echo $cmd . "\n";
    
$result system($cmd);
    if (!empty(
$result)) {
        echo 
$pkg." is already downloaded\n";
        
$counter += 1;
    } else {
        
$content .= $pkg "\n";
    }
}

file_put_contents('pkg_list.txt'$content);

The script works on subdirectory structure generated by previous script (ubuntu-repo-split-files-into-subdirectories.php). Before calling the script we have to set proper value to $repo_dir variable. We call it as follows: php5 ubuntu-repo-check-if-downloaded.php

Sometimes it happens that downloaded packages are files of 0 size. We maust remove them and download again. We do it by using script ubuntu-repo-remove-empty-files.php:


<?php

$dir 
"/mnt/sdb1/repository";

$files scandir($dir);
foreach (
$files as $file) {
    if (
is_file("$dir/$file")) {
        if (
filesize("$dir/$file") == 0) {
            echo 
"Removing file: $dir/$file\n";
            
unlink("$dir/$file");
        }
    }
    if (
is_dir("$dir/$file")) {
        
$pkgs scandir("$dir/$file");
        foreach (
$pkgs as $pkg) {
            if (
is_file("$dir/$file/$pkg")) {
                if (
filesize("$dir/$file/$pkg") == 0) {
                       echo 
"Removing file: $dir/$file/$pkg\n";
                    
unlink("$dir/$file/$pkg");
                }
            }
        }
    }
}

Before we run the script we must set proper value to $repo_dir variable. Next we run the script: php5 ubuntu-repo-remove-empty-files.php After removing empty files we must refresh packages for download list and run downloading process.

When all files are downloaded we make repository of them. We create directory /var/www/repository and inside it subdirectories /var/www/repository/repo_01 .. /var/www/repository/repo_09. Each of subdirectories repo_0x should contain symlinks to three one letter directories containing packages. For example /var/www/repository/repo_01 directory should contain symlinks to a, b, c directories. /var/www/repository/repo_02 directory should contain symlinks to d, e, f directories, and so on.

Now we enter each of /var/www/repository/repo_01 .. /var/www/repository/repo_09 directories and run following command inside them: dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz Be prepared that this is very time consuming process.

When we have all Packages.gz files, we configure client machine to get packages from our web server. Add following lines to the /etc/apt/sources.list file (near the top) on client machine: deb http://localhost/repository/repo_01 /repo_01
deb http://localhost/repository/repo_02 /repo_02
...
deb http://localhost/repository/repo_02 /repo_09