203 lines
4.8 KiB
Markdown
203 lines
4.8 KiB
Markdown
# RabbitMQ Integration Guide
|
|
|
|
## Overview
|
|
|
|
This project now uses RabbitMQ for asynchronous message queue processing. The architecture follows the **producer-consumer pattern**:
|
|
|
|
- **Senders (Producers)**: API endpoints send messages to RabbitMQ queues
|
|
- **Receivers (Consumers)**: Separate consumer processes listen to queues and save to databases
|
|
|
|
## Architecture
|
|
|
|
### Message Flow
|
|
|
|
```
|
|
API Endpoint → Service → RabbitMQ Queue → Consumer → Database
|
|
```
|
|
|
|
### Queues
|
|
|
|
- **users_queue**: Receives user events (create, update, delete)
|
|
- **reports_queue**: Receives report events (create, update_visibility, delete)
|
|
|
|
## Setup and Configuration
|
|
|
|
### 1. Install Dependencies
|
|
|
|
```bash
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
This includes the `pika` package for RabbitMQ communication.
|
|
|
|
### 2. Start RabbitMQ
|
|
|
|
Ensure RabbitMQ is running on your system:
|
|
|
|
```bash
|
|
# Using Docker
|
|
docker run -d --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:3-management
|
|
|
|
# Or using local installation
|
|
rabbitmq-server
|
|
```
|
|
|
|
## Running the Application
|
|
|
|
### Start the API Server
|
|
|
|
```bash
|
|
cd src
|
|
python main.py
|
|
```
|
|
|
|
The API will be available at `http://localhost:8000`
|
|
|
|
### Start Consumers
|
|
|
|
In separate terminal windows, run the consumers:
|
|
|
|
#### User Consumer
|
|
```bash
|
|
cd src
|
|
python -m consumers.user_consumer
|
|
```
|
|
|
|
#### Report Consumer
|
|
```bash
|
|
cd src
|
|
python -m consumers.report_consumer
|
|
```
|
|
|
|
## Usage Examples
|
|
|
|
### Creating a User
|
|
|
|
```bash
|
|
curl -X POST http://localhost:8000/users/ \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"nombre": "John",
|
|
"apellido": "Doe",
|
|
"email": "john@example.com",
|
|
"fecha_nacimiento": "1990-01-01T00:00:00",
|
|
"url_foto_perfil": "http://example.com/photo.jpg",
|
|
"biografia": "A test user"
|
|
}'
|
|
```
|
|
|
|
**Response** (Immediate):
|
|
```json
|
|
{
|
|
"status": "queued",
|
|
"message": "Usuario enviado a cola para procesamiento",
|
|
"email": "john@example.com"
|
|
}
|
|
```
|
|
|
|
The user will be saved to the database by the User Consumer.
|
|
|
|
### Creating a Report
|
|
|
|
```bash
|
|
curl -X POST http://localhost:8000/reports/ \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"id_usuario": 1,
|
|
"tipo_reporte": 1,
|
|
"descripcion": "Issue description",
|
|
"ubicacion": "Location info"
|
|
}'
|
|
```
|
|
|
|
**Response** (Immediate):
|
|
```json
|
|
{
|
|
"status": "queued",
|
|
"message": "Reporte enviado a cola para procesamiento",
|
|
"id_reporte": "uuid-string"
|
|
}
|
|
```
|
|
|
|
The report will be saved to the database by the Report Consumer.
|
|
|
|
## Message Formats
|
|
|
|
### User Event Messages
|
|
|
|
```python
|
|
{
|
|
"event_type": "user.create|user.update|user.delete",
|
|
"user_id": Optional[int],
|
|
"nombre": Optional[str],
|
|
"apellido": Optional[str],
|
|
"email": Optional[str],
|
|
"fecha_nacimiento": Optional[str], # ISO format
|
|
"fecha_creacion": Optional[str], # ISO format
|
|
"calificacion": Optional[float],
|
|
"numero_reportes": Optional[int],
|
|
"url_foto_perfil": Optional[str],
|
|
"biografia": Optional[str]
|
|
}
|
|
```
|
|
|
|
### Report Event Messages
|
|
|
|
```python
|
|
{
|
|
"event_type": "report.create|report.update_visibility|report.delete",
|
|
"id_reporte": Optional[str],
|
|
"id_usuario": Optional[int],
|
|
"tipo_reporte": Optional[int],
|
|
"descripcion": Optional[str],
|
|
"ubicacion": Optional[str],
|
|
"visibilidad": Optional[float],
|
|
"fecha_creacion": Optional[str], # ISO format
|
|
"penalize_author": Optional[bool]
|
|
}
|
|
```
|
|
|
|
## Consumer Implementation Details
|
|
|
|
### User Consumer (`src/consumers/user_consumer.py`)
|
|
|
|
Processes three types of user events:
|
|
|
|
1. **CREATE**: Saves a new user to the database
|
|
2. **UPDATE**: Updates existing user fields
|
|
3. **DELETE**: Removes a user from the database
|
|
|
|
### Report Consumer (`src/consumers/report_consumer.py`)
|
|
|
|
Processes three types of report events:
|
|
|
|
1. **CREATE**: Saves a new report to MongoDB and increments user's report counter
|
|
2. **UPDATE_VISIBILITY**: Updates report visibility and optionally penalizes the author
|
|
3. **DELETE**: Removes a report from the database
|
|
|
|
## Benefits of This Architecture
|
|
|
|
1. **Asynchronous Processing**: API responds immediately without waiting for database operations
|
|
2. **Scalability**: Consumers can be scaled independently
|
|
3. **Reliability**: Messages are persistent and won't be lost
|
|
4. **Decoupling**: Services are decoupled from database operations
|
|
5. **Message Ordering**: FIFO guarantee ensures operations are processed in order
|
|
|
|
## Error Handling
|
|
|
|
- Messages are acknowledged only after successful processing
|
|
- Failed messages are automatically requeued for retry
|
|
- All operations are logged for debugging and monitoring
|
|
|
|
## Database Compatibility
|
|
|
|
- **Users**: MySQL (via SQLAlchemy)
|
|
- **Reports**: MongoDB
|
|
|
|
## Future Enhancements
|
|
|
|
- Add retry policies with exponential backoff
|
|
- Implement dead-letter queues for failed messages
|
|
- Add message monitoring and analytics
|
|
- Implement distributed transaction handling
|