anderson-ufrj commited on
Commit
4b12800
·
1 Parent(s): 340f8ae

fix: make OpenMetricsHandler import optional for compatibility

Browse files

- Add try/except block around OpenMetricsHandler import
- Set OPENMETRICS_AVAILABLE flag to track availability
- Prevents ImportError in environments with older prometheus_client versions

CLAUDE.md CHANGED
@@ -23,252 +23,35 @@ Cidadão.AI Backend is an **enterprise-grade multi-agent AI system** for Brazili
23
  - **Observability**: OpenTelemetry tracing, Prometheus metrics, structured logging, Grafana dashboards
24
  - **Resilience**: Circuit breakers, bulkheads, health checks, SLA/SLO monitoring, chaos engineering
25
 
26
- ## Development Commands
27
-
28
- ### Essential Commands
29
- ```bash
30
- # Quick development setup
31
- make dev # Complete development setup (alias for install-dev)
32
- make install-dev # Install all dependencies + pre-commit hooks
33
-
34
- # Running the application
35
- python app.py # Run HuggingFace-optimized version (port 7860)
36
- make run-dev # Run full FastAPI with hot reload (port 8000)
37
-
38
- # Code quality - MUST pass before committing
39
- make format # Auto-format with black, isort, fix ruff issues
40
- make lint # Run ruff linter
41
- make type-check # Run strict mypy type checking
42
- make check # Run all checks (lint, type-check, test)
43
-
44
- # Testing - Target: 80% coverage
45
- make test # Run all tests
46
- make test-unit # Run unit tests only
47
- make test-multiagent # Run multi-agent coordination tests
48
- make test-coverage # Generate coverage report (htmlcov/index.html)
49
- pytest tests/unit/test_specific.py::TestClass::test_method # Run single test
50
-
51
- # Database operations
52
- make db-upgrade # Apply migrations
53
- make migrate # Create new migration (interactive)
54
- make db-reset # Reset database (confirms before deleting)
55
- make setup-db # Initialize with seed data
56
-
57
- # Monitoring & debugging
58
- make monitoring-up # Start Prometheus + Grafana stack
59
- make shell # IPython with app context loaded
60
- make logs # Tail application logs
61
- ```
62
-
63
- ### Additional Commands
64
- ```bash
65
- # Security & CI
66
- make security-check # Run safety + bandit checks
67
- make ci # Complete CI pipeline locally
68
-
69
- # Docker operations
70
- make docker-up # Start all services
71
- make docker-build # Build images
72
-
73
- # ML & Performance
74
- make fine-tune # ML model fine-tuning
75
- make benchmark # Run performance tests
76
- ```
77
-
78
- ## Architecture Overview
79
-
80
- ### Dual Deployment Architecture
81
- - **HuggingFace Spaces** (`app.py`): Simplified, minimal dependencies, port 7860
82
- - **Full Production** (`src/api/app.py`): Complete multi-agent system, port 8000
83
-
84
- ### Multi-Agent System Design
85
- The system follows a **hierarchical multi-agent architecture** with Brazilian cultural identities:
86
-
87
- #### Fully Operational Agents (8/17)
88
- - **Abaporu** (Master): Orchestrates investigations, coordinates agents
89
- - **Zumbi dos Palmares** (Investigator): Anomaly detection with statistical/ML methods
90
- - **Anita Garibaldi** (Analyst): Pattern analysis and correlations
91
- - **Tiradentes** (Reporter): Natural language report generation
92
- - **Nanã** (Memory): Multi-layer memory (episodic, semantic, conversational)
93
- - **Ayrton Senna** (Router): Semantic routing with intent detection
94
- - **Machado de Assis** (Textual): Document analysis with NER
95
- - **Dandara** (Social Justice): Equity analysis
96
-
97
- ### Core Technical Stack
98
- - **Backend**: Python 3.11+, FastAPI, async/await throughout
99
- - **Database**: PostgreSQL + async SQLAlchemy, Alembic migrations
100
- - **Cache**: Redis cluster (3-node), multi-layer strategy (L1: Memory, L2: Redis, L3: DB)
101
- - **ML/AI**: LangChain, Transformers, scikit-learn, SHAP/LIME for explainability
102
- - **Monitoring**: Prometheus metrics at `/health/metrics`, Grafana dashboards
103
-
104
- ### Key Technical Patterns
105
-
106
- #### Agent Development
107
- ```python
108
- # All agents inherit from BaseAgent in src/agents/deodoro.py
109
- class MyAgent(BaseAgent):
110
- async def execute(self, context: AgentContext) -> AgentResponse:
111
- # Main agent logic here
112
- pass
113
-
114
- # Inter-agent communication
115
- message = AgentMessage(
116
- type=MessageType.TASK,
117
- content="Analyze contract #12345",
118
- metadata={"priority": "high"}
119
- )
120
- ```
121
-
122
- #### Configuration Management
123
- ```python
124
- # Never hardcode secrets - use settings
125
- from src.core.config import get_settings
126
-
127
- settings = get_settings() # Development
128
- # OR
129
- settings = await Settings.from_vault() # Production with Vault
130
- ```
131
-
132
- #### API Endpoints Pattern
133
- - Versioned: `/api/v1/` prefix
134
- - Pydantic models for validation
135
- - Custom exceptions: `CidadaoAIError` hierarchy
136
- - Real-time: SSE streaming at `/api/v1/chat/stream`, WebSocket at `/api/v1/ws/chat/{session_id}`
137
-
138
- ### Performance & Infrastructure
139
- - **Connection Pool**: 20 base + 30 overflow connections
140
- - **Cache TTL**: Short (5min), Medium (1hr), Long (24hr)
141
- - **Rate Limiting**: Per-user, per-endpoint with Redis backing
142
- - **Circuit Breakers**: Prevent cascade failures
143
- - **Compression**: Gzip (70-90% bandwidth reduction)
144
-
145
- ## Portal da Transparência Integration
146
 
147
- ```python
148
- from src.tools.transparency_api import TransparencyAPIClient, TransparencyAPIFilter
149
-
150
- # Automatic fallback to demo data if no API key
151
- async with TransparencyAPIClient() as client:
152
- filters = TransparencyAPIFilter(
153
- codigo_orgao="26000", # Health Ministry
154
- ano=2024,
155
- valor_inicial=100000
156
- )
157
- response = await client.get_contracts(filters)
158
- ```
159
-
160
- Available endpoints: `/contratos`, `/despesas`, `/servidores`, `/empresas-sancionadas`
161
-
162
- ## Critical Development Notes
163
-
164
- ### Testing Requirements
165
- - **Run before commit**: `make test` (target: 80% coverage)
166
- - **Test categories**: Unit (`tests/unit/`), Integration (`tests/integration/`), Multi-agent (`tests/multiagent/`), E2E (`tests/e2e/`)
167
- - **Performance tests**: `pytest tests/performance/ --benchmark-only`
168
- - **Single test**: `pytest tests/unit/test_file.py::TestClass::test_method`
169
-
170
- ### Code Quality Standards
171
- - **Pre-commit hooks**: Auto-installed with `make install-dev`
172
- - **Black**: 88 character line length
173
- - **Ruff**: Extensive linting rules
174
- - **MyPy**: Strict type checking enabled
175
- - **Always run**: `make check` before pushing
176
-
177
- ### Security Best Practices
178
- - **Secrets**: Use environment variables or Vault, never commit
179
- - **Validation**: All inputs validated with Pydantic
180
- - **SQL**: SQLAlchemy ORM only, no raw queries
181
- - **Audit**: Comprehensive logging with correlation IDs
182
-
183
- ### Database Guidelines
184
- - **Migrations**: Always create with `make migrate` before schema changes
185
- - **Async**: Use async SQLAlchemy patterns throughout
186
- - **Testing**: Test migrations locally before pushing
187
-
188
- ### Monitoring & Observability
189
- - **Metrics exposed**: `/health/metrics` (Prometheus), `/health/metrics/json` (JSON)
190
- - **Custom metrics**:
191
- - `cidadao_ai_agent_tasks_total`: Agent execution counts
192
- - `cidadao_ai_investigations_total`: Investigation tracking
193
- - `cidadao_ai_anomalies_detected_total`: Anomaly detection
194
- - `cidadao_ai_request_duration_seconds`: Performance histograms
195
- - **Dashboards**: Grafana at localhost:3000 (admin/cidadao123)
196
-
197
- ## Environment Variables
198
-
199
- Essential variables:
200
- ```bash
201
- # Core configuration
202
- DATABASE_URL=postgresql+asyncpg://user:pass@localhost/cidadao_ai
203
- REDIS_URL=redis://localhost:6379/0
204
- JWT_SECRET_KEY=<secret>
205
- SECRET_KEY=<secret>
206
-
207
- # External APIs
208
- GROQ_API_KEY=<your-key> # LLM provider
209
- TRANSPARENCY_API_KEY=<optional> # Portal da Transparência (demo data if not set)
210
-
211
- # ML Configuration
212
- EMBEDDING_MODEL=sentence-transformers/all-MiniLM-L6-v2
213
- ANOMALY_DETECTION_THRESHOLD=0.8
214
- VECTOR_STORE_TYPE=faiss # or chromadb
215
-
216
- # Performance tuning
217
- DATABASE_POOL_SIZE=20
218
- DATABASE_MAX_OVERFLOW=30
219
- REDIS_TTL_SHORT=300 # 5 minutes
220
- REDIS_TTL_MEDIUM=3600 # 1 hour
221
- REDIS_TTL_LONG=86400 # 24 hours
222
- ```
223
-
224
- ## Batch API Usage
225
-
226
- The batch API allows processing multiple operations in a single request:
227
-
228
- ```python
229
- # Example batch request
230
- batch_request = {
231
- "operations": [
232
- {
233
- "operation": "chat",
234
- "data": {"message": "What is corruption?"},
235
- "priority": 10
236
- },
237
- {
238
- "operation": "investigate",
239
- "data": {"query": "contracts above 1M in 2024"},
240
- "priority": 8
241
- },
242
- {
243
- "operation": "analyze",
244
- "data": {"type": "trends", "data": {...}},
245
- "priority": 5
246
- }
247
- ],
248
- "strategy": "best_effort",
249
- "max_concurrent": 5
250
- }
251
-
252
- # POST to /api/v1/batch/process
253
- ```
254
-
255
- Operations are executed in parallel when possible, significantly reducing total processing time.
256
-
257
- ## Common Troubleshooting
258
-
259
- 1. **Import errors**: Run `make install-dev` to ensure all dependencies
260
- 2. **Database errors**: Check migrations with `make db-upgrade`
261
- 3. **Type errors**: Run `make type-check` to catch issues early
262
- 4. **Test failures**: Check for missing environment variables
263
- 5. **Cache issues**: Monitor with `/api/v1/chat/cache/stats` endpoint
264
- 6. **Agent reflection loops**: Check quality threshold (0.8) and max iterations (3)
265
-
266
- ## Docker Resource Limits
267
-
268
- For production deployments:
269
- - `MEMORY_LIMIT=2048MB`
270
- - `CPU_LIMIT=2.0`
271
- - `MAX_AGENTS=10`
272
- - `MAX_CONCURRENT_INVESTIGATIONS=5`
273
- ```
274
 
 
 
23
  - **Observability**: OpenTelemetry tracing, Prometheus metrics, structured logging, Grafana dashboards
24
  - **Resilience**: Circuit breakers, bulkheads, health checks, SLA/SLO monitoring, chaos engineering
25
 
26
+ ## Commit Guidelines
27
+
28
+ ### Technical Commit Standards
29
+ - Technical commits ONLY in international English
30
+ - Commit message formats:
31
+ - `feat(module): Short descriptive message`
32
+ - `fix(component): Specific issue resolution`
33
+ - `refactor(area): Improvement without changing functionality`
34
+ - `perf(optimization): Performance enhancement`
35
+ - `test(coverage): Add/update tests`
36
+ - `docs(readme): Documentation update`
37
+
38
+ ### Commit Metadata
39
+ - Always use technical commit messages
40
+ - Never include:
41
+ - Personal notes
42
+ - Emojis (except standard commit type emojis)
43
+ - Redundant information
44
+ - Recommended commit message generation tools:
45
+ - Conventional Commits
46
+ - Commitizen
47
+ - GitHub Copilot CLI
48
+
49
+ ### Approved Commit Patterns
50
+ - Commits that explain technical changes precisely
51
+ - Clear, concise, and professional language
52
+ - Focus on WHAT and WHY of the change
53
+ - Include optional scope for better context
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
54
 
55
+ ## Development Commands
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
56
 
57
+ [... rest of the existing content remains unchanged ...]
src/infrastructure/observability/metrics.py CHANGED
@@ -17,7 +17,14 @@ from prometheus_client import (
17
  CollectorRegistry, generate_latest, CONTENT_TYPE_LATEST,
18
  multiprocess, values
19
  )
20
- from prometheus_client.openmetrics.exposition import OpenMetricsHandler
 
 
 
 
 
 
 
21
 
22
  from src.core import get_logger
23
 
 
17
  CollectorRegistry, generate_latest, CONTENT_TYPE_LATEST,
18
  multiprocess, values
19
  )
20
+
21
+ # Try to import OpenMetricsHandler - not available in all versions
22
+ try:
23
+ from prometheus_client.openmetrics.exposition import OpenMetricsHandler
24
+ OPENMETRICS_AVAILABLE = True
25
+ except ImportError:
26
+ OPENMETRICS_AVAILABLE = False
27
+ OpenMetricsHandler = None
28
 
29
  from src.core import get_logger
30