First we make sure that we have php5 command line interpreter installed:
sudo apt-get install php5-cli
Next we obtain list of all deb packages in Ubuntu repository.
apt-cache pkgnames > pkg_list.txt
We get the list in pkg_list.txt file.
When we have packages list we can start downloading them from Ubuntu server. We do it by using following php5 script ubuntu-repo-download.php:
<?php
$lines = file('pkg_list.txt');
foreach ($lines as $pkg) {
$pkg = trim($pkg);
$cmd = "apt-get download $pkg";
echo $cmd . "\n";
system($cmd);
}
The script downloads all listed packages to a current directory. We call the script by typing in command line:
php5 ubuntu-repo-download.php
When large amount of packages is downloaded we can stop download process by pressing Ctrl+C. Now we can divide packages into the one letter subdirectories. Script ubuntu-repo-split-files-into-subdirectories.php helps us with the task:
<?php
$srcdir = "/mnt/sdb1/repository";
$destdir = "/mnt/sdb1/repository";
if (is_dir($srcdir)) {
$files = scandir($srcdir);
foreach ($files as $file) {
if (is_file("$srcdir/$file")) {
$letter = $file{0};
if (!is_dir("$destdir/$letter")) {
mkdir("$destdir/$letter");
}
if (is_dir("$destdir/$letter")) {
echo "Moving file: $file\n";
rename("$srcdir/$file", "$destdir/$letter/$file");
}
}
}
}
Before we run the script we have to set proper values to variables $srcdir and
$destdir. Both of the variables shoud be set to the downloaded packages' directory.
We run the script with following command:
php5 ubuntu-repo-split-files-into-subdirectories.php
Before continueing download process, we have to update packages list. This task is resolved by following script ubuntu-repo-check-if-downloaded.php:
<?php
$repo_dir = "/mnt/sdb1/repository";
$lines = file('pkg_list.txt');
$content = "";
$counter = 0;
foreach ($lines as $pkg) {
$pkg = trim($pkg);
$letter = $pkg{0};
$cmd = "ls " . $repo_dir . "/" . $letter . "/" . $pkg . "_* 2>/dev/null";
//echo $cmd . "\n";
$result = system($cmd);
if (!empty($result)) {
echo $pkg." is already downloaded\n";
$counter += 1;
} else {
$content .= $pkg . "\n";
}
}
file_put_contents('pkg_list.txt', $content);
The script works on subdirectory structure generated by previous script (ubuntu-repo-split-files-into-subdirectories.php).
Before calling the script we have to set proper value to $repo_dir variable.
We call it as follows:
php5 ubuntu-repo-check-if-downloaded.php
Sometimes it happens that downloaded packages are files of 0 size. We maust remove them and download again. We do it by using script ubuntu-repo-remove-empty-files.php:
<?php
$dir = "/mnt/sdb1/repository";
$files = scandir($dir);
foreach ($files as $file) {
if (is_file("$dir/$file")) {
if (filesize("$dir/$file") == 0) {
echo "Removing file: $dir/$file\n";
unlink("$dir/$file");
}
}
if (is_dir("$dir/$file")) {
$pkgs = scandir("$dir/$file");
foreach ($pkgs as $pkg) {
if (is_file("$dir/$file/$pkg")) {
if (filesize("$dir/$file/$pkg") == 0) {
echo "Removing file: $dir/$file/$pkg\n";
unlink("$dir/$file/$pkg");
}
}
}
}
}
Before we run the script we must set proper value to $repo_dir variable.
Next we run the script:
php5 ubuntu-repo-remove-empty-files.php
After removing empty files we must refresh packages for download list and run downloading process.
When all files are downloaded we make repository of them. We create directory /var/www/repository and inside it subdirectories /var/www/repository/repo_01 .. /var/www/repository/repo_09. Each of subdirectories repo_0x should contain symlinks to three one letter directories containing packages. For example /var/www/repository/repo_01 directory should contain symlinks to a, b, c directories. /var/www/repository/repo_02 directory should contain symlinks to d, e, f directories, and so on.
Now we enter each of /var/www/repository/repo_01 .. /var/www/repository/repo_09 directories and
run following command inside them:
dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz
Be prepared that this is very time consuming process.
When we have all Packages.gz files, we configure client machine to get packages from our web server.
Add following lines to the /etc/apt/sources.list file (near the top) on client machine:
deb http://localhost/repository/repo_01 /repo_01
deb http://localhost/repository/repo_02 /repo_02
...
deb http://localhost/repository/repo_02 /repo_09