What you'll need
- Podman
- Docker
- Knowledge (basic) Linux
Limits
File Size Limits: Files larger than 100 MB are blocked, and files over 50 MiB will trigger a warning. For large files (like images, videos, or datasets), you must use Git Large File Storage
Only good for pics and text unless if you use git lfs, or use the method below.
Only issue for most people:
Pricing Options
Git LFS is open-source and free to use, but remote hosting services have limits. For instance, GitHub and Bitbucket offer a set amount of storage and bandwidth for free, with options to purchase additional capacity (e.g., 100 GB for $10/month on Bitbucket).
For radicle:
Git LFS (Large File Storage) and
Radicle are related in that they both interact with the Git ecosystem, but they are separate tools with different purposes. Radicle is currently considering adding support for LFS or similar solutions like git-annex for handling large files.
Current Status regarding LFS: Radicle uses standard Git for data transfer and is designed to handle moderately-sized binaries. Its developers have stated they are considering adding support for solutions like Git LFS or git-annex in the future to better manage very large files, such as AI models and datasets, within the decentralized network.
Docker compose
Input these in the docker image with the gui of your choice.
mkdir -p ./targetdir && cp /mnt/repos/ ./targetdir && cd targetdir && git config --global user.email "[email protected]" && git config --global user.name "name" && git init && git add . && git commit -m "."
|
For videos:
This is a lot slower but it much better
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 | #!/usr/bin/env bash
mkdir -p Output
# Check if Videos directory exists and has files
if [ ! -d "Videos" ]; then
echo "Error: Videos/ directory not found!"
exit 1
fi
# Store original directory
original_dir="$(pwd)"
for f in Videos/*.mp4 Videos/*.m4v; do
# Skip if no matching files found
[ ! -f "$f" ] && continue
echo "Processing: $f"
# Base name without extension, used as "title"
base="$(basename "$f" .mp4)"
base="${base%.m4v}" # Handle both .mp4 and .m4v extensions
outdir="Output/$base"
mkdir -p "$outdir/images"
############################
# 1) Split video to images #
############################
echo " Extracting frames..."
ffmpeg -i "$f" -q:v 2 "$outdir/images/frame_%06d.jpg"
# Check if frames were created
frame_count=$(ls "$outdir/images"/*.jpg 2>/dev/null | wc -l)
if [ "$frame_count" -eq 0 ]; then
echo " No frames extracted from $f, skipping..."
continue
fi
echo " Extracted $frame_count frames"
###############################
# 2) Compress frames and ensure even dimensions #
###############################
echo " Compressing frames..."
cd "$outdir/images"
# First, resize all images to 40% and reduce quality
echo " Resizing all images..."
mogrify -resize 35% -quality 40% *.jpg
# Then, adjust any images with odd dimensions to be even
echo " Ensuring all images have even dimensions (H.264 requirement)..."
for image_file in *.jpg; do
# Get current width and height
width=$(identify -format "%w" "$image_file" 2>/dev/null)
height=$(identify -format "%h" "$image_file" 2>/dev/null)
# Skip if we can't read dimensions
[ -z "$width" ] || [ -z "$height" ] && continue
# Check if dimensions need adjustment
if [ $((width % 2)) -eq 1 ] || [ $((height % 2)) -eq 1 ]; then
# Make width even
if [ $((width % 2)) -eq 1 ]; then
width=$((width - 1))
fi
# Make height even
if [ $((height % 2)) -eq 1 ]; then
height=$((height - 1))
fi
# Apply the adjusted dimensions
mogrify -resize "${width}x${height}!" "$image_file"
fi
done
echo " Compression complete!"
############################
# 3) Check if video has audio, then process accordingly #
############################
# Return to original directory for audio check
cd "$original_dir"
has_audio=$(ffprobe -v quiet -select_streams a:0 -show_entries stream=codec_name -of csv=p=0 "$f" 2>/dev/null | grep -c .)
if [ "$has_audio" -gt 0 ]; then
echo " Video has audio - creating video-only + merging audio..."
# Re-render video from compressed frames (no audio)
# Use the concat demuxer instead of glob pattern
cd "$outdir/images"
ls frame_*.jpg | sort > filelist.txt
sed -i 's/^/file /' filelist.txt
cd "$original_dir"
ffmpeg -f concat -safe 0 -r 25 -i "$outdir/images/filelist.txt" -c:v libx264 -crf 30 -preset ultrafast -pix_fmt yuv420p "$outdir/${base}_video_only.mp4"
# Extract and merge audio
ffmpeg -i "$f" -vn -c:a libmp3lame -q:a 4 "$outdir/${base}.mp3"
ffmpeg -i "$outdir/${base}_video_only.mp4" -i "$outdir/${base}.mp3" -c:v copy -c:a aac -map 0:v:0 -map 1:a:0 -shortest "$outdir/${base}_recompressed.mp4"
# Clean up
rm "$outdir/${base}_video_only.mp4" "$outdir/${base}.mp3" "$outdir/images/filelist.txt"
else
echo " Video has no audio - direct re-encode..."
# Use concat demuxer instead of glob pattern
cd "$outdir/images"
ls frame_*.jpg | sort > filelist.txt
sed -i 's/^/file /' filelist.txt
cd "$original_dir"
# Direct re-encode to final video (no audio processing needed)
ffmpeg -f concat -safe 0 -r 25 -i "$outdir/images/filelist.txt" -c:v libx264 -crf 30 -preset ultrafast -pix_fmt yuv420p "$outdir/${base}_recompressed.mp4"
# Clean up
rm "$outdir/images/filelist.txt"
fi
echo " Done: $outdir/${base}_recompressed.mp4"
echo ""
done
echo "Processing complete!"
|
Also to directly reduce the video
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 | #!/usr/bin/env bash
# Configuration
TARGET_SIZE_MB=80
OUTPUT_DIR="Output"
# Create output directory
mkdir -p "$OUTPUT_DIR"
# Check if Videos directory exists
if [ ! -d "Videos" ]; then
echo "Error: Videos/ directory not found!"
exit 1
fi
echo "Target output size: ${TARGET_SIZE_MB}MB per video"
echo ""
# Process each video
for input_file in Videos/*.mp4 Videos/*.m4v Videos/*.mov Videos/*.avi Videos/*.mkv; do
# Skip if no matching files (nullglob behavior)
[ ! -f "$input_file" ] 2>/dev/null && continue
filename=$(basename "$input_file")
basename="${filename%.*}"
output_file="${OUTPUT_DIR}/${basename}_compressed.mp4"
echo "Processing: ${filename}"
# Get video duration in seconds (handle errors)
duration=$(ffprobe -v error -show_entries format=duration -of default=noprint_wrappers=1:nokey=1 "$input_file" 2>/dev/null)
# Extract integer seconds safely
if [ -n "$duration" ]; then
duration_int=$(echo "$duration" | cut -d. -f1)
# Ensure duration is at least 1 second
if [ -z "$duration_int" ] || [ "$duration_int" -lt 1 ]; then
duration_int=1
fi
else
duration_int=60 # Default if can't determine
fi
duration="$duration_int"
# Calculate target bitrate for 80MB (in kbps)
# Formula: (target_size * 8192) / duration
target_bitrate=$(( (TARGET_SIZE_MB * 8192) / duration ))
# Apply minimum and maximum bitrate limits (increased for better quality)
min_bitrate=200 # Increased from 100 kbps
max_bitrate=3000 # Increased from 2000 kbps
# Use safer comparisons
if [ "$target_bitrate" -lt "$min_bitrate" ]; then
target_bitrate="$min_bitrate"
fi
if [ "$target_bitrate" -gt "$max_bitrate" ]; then
target_bitrate="$max_bitrate"
fi
echo " Duration: ${duration}s"
echo " Target bitrate: ${target_bitrate}kbps"
# Check if video has audio
has_audio=$(ffprobe -v quiet -select_streams a -show_entries stream=codec_type -of csv=p=0 "$input_file" 2>/dev/null | grep -c audio)
# Build FFmpeg command with better quality settings
ffmpeg_cmd="ffmpeg -i \"$input_file\" -c:v libx264"
# Better quality video settings
ffmpeg_cmd="$ffmpeg_cmd -preset medium" # Better compression efficiency
ffmpeg_cmd="$ffmpeg_cmd -crf 28" # Better quality (lower number)
ffmpeg_cmd="$ffmpeg_cmd -maxrate ${target_bitrate}k"
ffmpeg_cmd="$ffmpeg_cmd -bufsize $((target_bitrate * 2))k"
# Scale to higher resolution (854x480 instead of 640x360)
ffmpeg_cmd="$ffmpeg_cmd -vf \"scale='min(854,iw)':'min(480,ih)':force_original_aspect_ratio=decrease,pad=ceil(iw/2)*2:ceil(ih/2)*2\""
# Better frame rates
if [ "$duration" -gt 300 ]; then
ffmpeg_cmd="$ffmpeg_cmd -r 24" # Increased from 15 fps
else
ffmpeg_cmd="$ffmpeg_cmd -r 30" # Increased from 24 fps
fi
# Better audio settings
if [ "$has_audio" -gt 0 ]; then
ffmpeg_cmd="$ffmpeg_cmd -c:a aac -b:a 64k -ac 2" # Stereo, better bitrate
else
ffmpeg_cmd="$ffmpeg_cmd -an"
fi
# Output file
ffmpeg_cmd="$ffmpeg_cmd -y \"$output_file\""
echo " Executing: ffmpeg compression..."
eval "$ffmpeg_cmd 2>/dev/null"
# Verify output
if [ -f "$output_file" ]; then
# Get file size in bytes (compatible with both macOS and Linux)
if command -v stat >/dev/null 2>&1; then
# Try different stat formats
output_size=$(stat -f%z "$output_file" 2>/dev/null || stat -c%s "$output_file" 2>/dev/null)
else
# Fallback to wc
output_size=$(wc -c < "$output_file" 2>/dev/null || echo "0")
fi
# Calculate MB with floating point for small files
if [ -n "$output_size" ] && [ "$output_size" -gt 0 ]; then
# Use awk for precise calculation
output_size_mb=$(echo "$output_size / 1048576" | bc -l 2>/dev/null || echo "scale=2; $output_size / 1048576" | bc 2>/dev/null)
# Format to 2 decimal places
if command -v bc >/dev/null 2>&1; then
output_size_mb_formatted=$(printf "%.2f" "$output_size_mb")
echo " Output: ${output_size_mb_formatted}MB"
# Adjust if file is significantly smaller than target
if [ $(echo "$output_size_mb < ($TARGET_SIZE_MB * 0.8)" | bc) -eq 1 ]; then
echo " File is smaller than expected, increasing quality..."
temp_file="${output_file}.temp"
# Recompress with better quality
ffmpeg -i "$input_file" -c:v libx264 -preset medium -crf 24 -vf "scale='min(1280,iw)':'min(720,ih)':force_original_aspect_ratio=decrease" -r 30 -c:a aac -b:a 128k -ac 2 -y "$temp_file" 2>/dev/null
if [ -f "$temp_file" ]; then
mv "$temp_file" "$output_file"
# Get new size
if command -v stat >/dev/null 2>&1; then
final_size=$(stat -f%z "$output_file" 2>/dev/null || stat -c%s "$output_file" 2>/dev/null)
else
final_size=$(wc -c < "$output_file" 2>/dev/null)
fi
if [ -n "$final_size" ]; then
final_size_mb=$(echo "scale=2; $final_size / 1048576" | bc 2>/dev/null)
echo " Final size: ${final_size_mb}MB (higher quality)"
fi
fi
fi
else
# Fallback without bc
output_size_mb_int=$((output_size / 1048576))
echo " Output: ${output_size_mb_int}MB"
fi
else
echo " Output: 0MB (file might be empty)"
fi
else
echo " Error: Failed to create output file"
fi
echo ""
done
echo "Processing complete!"
echo "Compressed files are in: ${OUTPUT_DIR}/"
if command -v ls >/dev/null 2>&1; then
ls -lh "$OUTPUT_DIR"/*.mp4 2>/dev/null | awk '{print $9 ": " $5}' 2>/dev/null || echo "No compressed files found"
else
echo "Files in output directory:"
find "$OUTPUT_DIR" -name "*.mp4" -exec echo " {}" \; 2>/dev/null || echo "No compressed files found"
fi
|
Frame extraction and audio extraction only
to recompile the frames
and a radicle shell script that nearly automated everything:
#!/bin/sh
mkdir ../init && cd ../init && cp -R ../targetdir/* ./
# Initialize git repository (in current directory /var/lib/radicle/init)
git init
# Set git global config
git config --global user.email "[email protected]"
git config --global user.name "Your Name"
# FIRST: Add Pictures and readme.md, then commit
git add Pictures readme.md
git commit -m "Initial commit: Pictures and readme"
# THEN: Run rad init
rad init
sleep 20
rad self
sleep 20
rad node status
sleep 10
# Process existing chk folders only (POSIX sh compatible)
set -- chk chk2 chk3 chk4 chk5 chk6 chk7 chk8 chk9 chk10 chk11 chk12 chk13 chk14 chk15 chk16 chk17 chk18 chk19 chk20 chk21 chk22 chk23 chk24 chk25 chk26 chk27 chk28 chk29 chk30 chk31 chk32 chk33 chk34 chk35 chk36
total=$#
i=1
while [ $i -le $total ]; do
chk_folder="output/$1"
echo "Processing $chk_folder ($i/$total)"
# Add the specific chk folder
git add "$chk_folder"
# Commit if there are changes
if git diff --cached --quiet; then
echo "- No changes in $chk_folder, skipping commit and push"
else
git commit -m "."
echo "✓ Committed $chk_folder"
# Push immediately after each commit
git push rad
echo "🚀 Pushed $chk_folder to rad"
sleep 10
rad sync
sleep 5
rad sync status
fi
shift # Move to next argument
# Wait 20 minutes between iterations (except after last)
if [ $i -lt $total ]; then
echo "⏳ Waiting 20 minutes before next..."
sleep 120
fi
i=`expr $i + 1`
done
echo "✅ Completed all $total chk folders (with per-commit pushes)!"
|
or use a LLM (like deepseek or perplexity) to cook something up for you in shell/python.
You may also use vlc or handbreak for videos.
black peter out.