Source operations

News Sources

Moroccan and worldwide publishers are kept as separate raw feeds and merged only at the AI event layer.

Local News sources

6 sources

SourceLanguageArticlesLast jobStatusActions

HHespresswww.hespress.comAR / FR71completed · 9/25/2026, 8:02:49 PMEnabled

MMédias24medias24.comFR / AR / EN0failed · 9/25/2026, 8:03:09 PMNeeds attention

LLe360fr.le360.maFR49completed · 9/25/2026, 8:03:09 PMEnabled

TTelQueltelquel.maFR29completed · 9/25/2026, 8:03:30 PMEnabled

LLe Matinlematin.maFR0failed · 9/25/2026, 8:03:31 PMNeeds attention

LLe Deskledesk.maFR0failed · 9/25/2026, 8:03:31 PMNeeds attention

Worldwide News sources

3 sources

SourceLanguageArticlesLast jobStatusActions

AAl Jazeerawww.aljazeera.comEN24completed · 9/25/2026, 8:03:31 PMEnabled

FFrance 24www.france24.comEN19completed · 9/25/2026, 8:03:33 PMEnabled

DDWwww.dw.comEN20completed · 9/25/2026, 8:03:38 PMEnabled

Shared extraction pipeline

DiscoveryPublisher feeds, latest streams and pagination are traversed per sourceArticle extractionJSON-LD, OpenGraph, RSS media, picture/lazy-image and semantic HTML fallbacksLanguagesArabic / French / English detection with UTF-8/legacy charset normalization and RTL renderingLimit per source1000 candidates per source · up to 30 archive pages · 24h window

Post-scrape AI pipeline

Store raw articles↓Cluster same event across sources↓Summarize + priority↓Translate AR / FR / EN↓Extract map locations + knowledge entities↓Reference every source article

Schedule

StatusAutomatic worker enabledIntervalEvery 30 minutesLast full run9/25/2026, 8:03:59 PMConcurrencyConfigurable with SCRAPER_CONCURRENCY

Operational note

Use conservative intervals, identify your crawler with SCRAPER_USER_AGENT, and review each publisher's current terms and robots directives before production deployment. Paywalled content is not bypassed.