Kopernik Pt.2 - Emir Tasan; Mert Bekirogullari; Taha Ucar; Mehmet Ozdemir; Teoman Kaman;

Week 1 02/18/2025

---

Summary

 Enhanced recording scripts, updated location/time settings, and automated data acquisition and scheduling. Explored alternative SDR solutions.

Accomplishments

Recording Scripts:  
 - Auto-creation of directories, termination of existing ⁠ rtl_fm ⁠ processes, absolute path usage, and detailed logging.

Script Updates: 
 - Updated ⁠ new.py ⁠ with corrected coordinates, timezone (EST), noise command, and PNG management.

SDR Integration:  
 - Connected SDR to gpredict, adjusted NOAA15 frequency, and tested gqrx as an alternative.

Data Pipeline:  
 - Created “Emir” folder, developed ⁠ n2yo.py ⁠ for data scraping, used ⁠ filter.py ⁠ to refine data, scheduled recordings with ⁠ schedule.py ⁠, and automated the workflow with ⁠ runall.py ⁠.

To Do

 -Test and debug record.py ⁠and automation loop.
 -Resolve remaining SDR and scheduling issues.
 -Fine-tune data processing and logging.

Week 2 02/25/2025

---

Summary

 This week, we focused on refining the satellite image processing pipeline, debugging existing scripts, and optimizing the scheduling system. We also improved the prediction and recording workflows to enhance data acquisition reliability.

Accomplishments

Recording & Processing:
 -Successfully tested and confirmed that record_passes generates valid WAV files.
 -Implemented and configured the predict software for our system, ensuring seamless execution.
 -Updated TLE files using schedule_all.sh and calculated satellite pass predictions.
 -Filtered valid satellite passes with an elevation range of 19 to 91 degrees using schedule_satellite.sh.
 -Successfully generated a WAV file with receive_and_process_satellite and processed it into an image.

Script Debugging & Enhancements:
 -Attempted to generate WAV files using the ephem library to predict satellite movement, but encountered issues and decided to halt further development.
 -Identified and resolved an issue in the script where wxotimg was not running in quiet mode, preventing popup interruptions.
 -Tested new WAV files and attempted fixes for mcir and hcvt, which were producing grayscale images instead of colored ones.
 -Confirmed that thermal and sea images generated correctly.

To Do

 -Investigate and resolve why mcir and hcvt images are in grayscale instead of their intended color formats.
 -Debug the missing map overlay in the generated images.
 -Continue optimizing the scheduling and recording processes for improved efficiency.

Week 3 03/04/2025

---

Summary

 This week, we focused on improving file organization, satellite data processing, and image management. We refined naming conventions, automated folder-based processing, and enhanced cleanup procedures. Additionally, we integrated new overlays and debugging efforts while ensuring periodic maintenance updates.

Accomplishments

  File Management & Processing:
 -Renamed the satellite to First Satellite for better file naming consistency.
 -Implemented a system to locate corresponding satellite folders and process the latest data in each folder.
 -Introduced an automatic cleanup mechanism that removes images older than 7 days.

 Image Enhancements & Debugging:
 -Added map overlay to the generated images.
 -Identified an issue with fog artifacts in some images that need further fixing.
 -Successfully made MCIR and MCIR-Precip images work.
 -Still need to work on HCVT processing.

 System & Automation Updates:
 -Integrated Teoman’s code into the project.
 -Updated files to automatically clean up every 10 days.
 -Updated the RTL command to improve functionality.
 -Reviewed the WP bot for performance and issues.
 -Adjusted satellite processing logic based on All Pass data.

To Do

 -Fix the fog issue in some images with the map overlay.
 -Continue working on making HCVT processing functional.
 -Further optimize satellite processing to improve efficiency.

Week 4 03/18/2025

---

Summary

-This week, we focused on improving satellite image quality, refining automation scripts, and troubleshooting map overlay issues. We enhanced noise reduction techniques, optimized processing efficiency, and explored alternative georeferencing solutions.

Accomplishments

Enhancements & Processing:

-Modified wp_bot for better functionality.
-Improved apply_enhancements.sh for noise reduction and overlay application.
-Enhanced file naming, logging, and cleanup mechanisms.

Image Processing & Debugging:

-Applied stronger noise reduction using ImageMagick.
-Addressed map overlay issues but faced compatibility problems with WXtoImg and Keplers-Updater.
-Switched to QGIS for georeferencing and noise reduction, improving image quality.

System & Automation Updates:

-Standardized overlay application across all processed images.
-Ensured better handling of temporary files to streamline workflows.

To Do

-Optimize script to process only new satellite files.
-Fine-tune noise reduction and overlay parameters for better image quality.

Week 5 03/25/2025

---

Summary

 This week, we focused on optimizing the image generation workflow by eliminating redundant processing across satellite folders. We also improved automation for image cleanup, updated the enhancement pipeline, and refined overlay and GIF display features. Significant progress was made in streamlining file management and testing web upload capabilities.

Accomplishments

File Management & Optimization:
 -Fixed the issue where all three NOAA folders were being processed redundantly.
 -Optimized the system to take the newest file from each individual folder instead of reprocessing all files across folders.
 -Cleaned up development files, test images, and unnecessary clutter to improve workspace efficiency.

Archiving & Automation:
 -Created a new script, move_images.sh, which is called after apply_enhancement.
 -Moves the newest images into the today folder.
 -Transfers older images into the archive_images folder.
 -Developed a script to automatically delete website images older than 7 days, ensuring up-to-date content on the web server.

Upload Testing:
 -Upload attempts were made to various NOAA satellite folders.
 -Upload to NOAA15 was successful.
 -Uploads to other folders failed and are currently under investigation.

File Order & Resizing:
 -Files are sorted to maintain a consistent frame order.
 -Images are resized using the high-quality LANCZOS filter.

Date/Time & Imagetype Extraction:
 -A regular expression is used to extract the 8-digit date and 6-digit time from the filename.
 -The imagetype is extracted as the second element in the filename, based on underscores.

Overlay Improvements:
 Overlays now include:
 -Satellite name (centered)
 -Time (top-right)
 -Imagetype (top-left)
 -Date (below the time)
 -Deprecated textsize() was replaced with textbbox() for more accurate text positioning.

GIF Output & Display:
 -GIFs are named using the format: satellite-yyyymmdd-hhmmss.gif.
 -Images are converted to ‘P’ mode with an adaptive palette for optimal GIF quality.
 -A Tkinter window displays the GIF on HDMI‑2 using the geometry setting "1920x1080+2560+0".
 -Escape key closes the window.
 -Proper handling of KeyboardInterrupt is implemented.

Cleanup:
 -Old GIFs are automatically deleted after the retention period to maintain storage efficiency.

To Do

 -Investigate and resolve upload failures for folders other than NOAA15.
 -Continue testing automated website cleanup and folder updating.
 -Monitor overlay formatting and display output for errors.
 -Refine archiving and file movement scripts if needed.

Week 6 04/01/2025

---

Summary

 This week, we improved the automated image display workflow by modifying the GIF generation process to ensure consistent behavior on the second monitor. Additional debugging was performed on hardware connectivity issues, and web upload functionality was enhanced with new metadata automation for the weather gallery.

Accomplishments:

GIF Display & Automation:
-Updated the gif_akb.py script to always display GIFs on the second monitor.
-The apply_enhancements script now calls gif_akb.py after move_images.sh is executed.
-Ensures that once a new image is detected and moved to the today folder, a new GIF is generated and displayed automatically.
-Old GIFs are automatically deleted before the new one is shown.

Hardware Debugging:
-Encountered a USB claim interface error -6 with the RTL2838 dongle.
-Used lsusb and lsusb -t to identify the device at port 1-1.4.
-Resolved the issue by unbinding the device using: sudo sh -c ‘echo “1-1.4” > /sys/bus/usb/drivers/usb/unbind’

Web Upload & Metadata Automation:
-Uploaded images to the weather folder successfully.
-Investigated the bias-t power option for further compatibility research.
-Updated wp_bot_upload with 5 new functions to automate gallery metadata:
-open_manage_galleries: Navigates to the Manage Galleries page.
-select_gal: Finds and clicks the edit link for the “weather” gallery.
-parse_filename: Extracts date, time, NOAA number, and sensor type from filenames.
-enter_text: Iterates through image rows and applies metadata to the ALT/Title and Description fields.
-click_save_changes: Scrolls and clicks the Save Changes button using JavaScript.
-The main function was updated to call these new steps after image uploads for full automation.

To Do

-Continue testing image display for edge cases and stability
-Finalize bias-t power option testing
-Monitor dongle behavior for recurring USB errors
-Expand metadata automation to other gallery folders if needed