forked from othman.suseno/Mail-Migrator
write code frame
This commit is contained in:
238
README.md
238
README.md
@@ -1,3 +1,239 @@
|
||||
# Mail-Migrator
|
||||
|
||||
Email migration tools
|
||||
Email migration tools untuk memindahkan mailbox IMAP antar server dengan dukungan resume, batch processing, dan insecure mode.
|
||||
|
||||
## Features
|
||||
|
||||
- ✅ **Single & Batch Migration**: Migrasi satu akun atau batch dari file CSV
|
||||
- ✅ **Resume Support**: Lanjutkan migrasi yang terputus menggunakan BoltDB state tracking
|
||||
- ✅ **Insecure Mode**: Terima self-signed certificates dengan flag `--insecure`
|
||||
- ✅ **Comprehensive Logging**: Log ke console dan file dengan timestamp
|
||||
- ✅ **Mailbox Detection**: Otomatis detect dan migrate semua mailbox/folder
|
||||
- ✅ **Batch Processing**: Proses message dalam batch untuk efisiensi
|
||||
- ✅ **Error Handling**: Robust error handling dengan retry logic
|
||||
|
||||
## Installation
|
||||
|
||||
### Build from Source
|
||||
|
||||
```bash
|
||||
git clone <repository-url>
|
||||
cd mail-migrator
|
||||
go mod tidy
|
||||
go build .
|
||||
```
|
||||
|
||||
### Download Binary
|
||||
|
||||
Download binary dari releases page atau build sendiri.
|
||||
|
||||
## Usage
|
||||
|
||||
### Single Account Migration
|
||||
|
||||
```bash
|
||||
# Basic migration
|
||||
./mail-migrator --src "user1:pass1@imap.old.com:993" \
|
||||
--dst "user1:pass1@imap.new.com:993"
|
||||
|
||||
# With insecure mode (trust self-signed certs)
|
||||
./mail-migrator --src "user1:pass1@imap.old.com:993" \
|
||||
--dst "user1:pass1@imap.new.com:993" \
|
||||
--insecure
|
||||
|
||||
# With resume support and logging
|
||||
./mail-migrator --src "user1:pass1@imap.old.com:993" \
|
||||
--dst "user1:pass1@imap.new.com:993" \
|
||||
--insecure \
|
||||
--resume \
|
||||
--log migration.log
|
||||
```
|
||||
|
||||
### Batch Migration
|
||||
|
||||
Buat file CSV dengan format: `source,destination`
|
||||
|
||||
**accounts.csv:**
|
||||
```csv
|
||||
user1:pass1@old.com:993,user1:newpass1@new.com:993
|
||||
user2:pass2@old.com:993,user2:newpass2@new.com:993
|
||||
user3:pass3@old.com:993,user3:newpass3@new.com:993
|
||||
```
|
||||
|
||||
```bash
|
||||
# Batch migration
|
||||
./mail-migrator --batch accounts.csv --insecure --resume --log batch.log
|
||||
```
|
||||
|
||||
## Command Line Options
|
||||
|
||||
| Flag | Description | Default |
|
||||
|------|-------------|---------|
|
||||
| `--src` | Source IMAP URL (user:pass@host:port) | - |
|
||||
| `--dst` | Destination IMAP URL (user:pass@host:port) | - |
|
||||
| `--batch` | CSV file with src,dst entries for batch migration | - |
|
||||
| `--insecure` | Allow insecure TLS (trust self-signed certs) | false |
|
||||
| `--resume` | Resume incomplete migrations | false |
|
||||
| `--state` | Path to state DB (Bolt) | state.db |
|
||||
| `--log` | Path to log file (optional) | - |
|
||||
|
||||
## URL Format
|
||||
|
||||
IMAP URL format: `username:password@hostname:port`
|
||||
|
||||
**Examples:**
|
||||
- `john:secret123@mail.example.com:993` (IMAPS)
|
||||
- `jane:mypass@imap.gmail.com:993` (Gmail)
|
||||
- `user@domain.com:password@mail.server.com:143` (IMAP plain)
|
||||
|
||||
**Port defaults:**
|
||||
- Port 993: IMAPS (TLS/SSL)
|
||||
- Port 143: IMAP (plain/STARTTLS)
|
||||
|
||||
## Resume Functionality
|
||||
|
||||
Aplikasi menggunakan BoltDB untuk menyimpan state migrasi:
|
||||
|
||||
- **State Key**: `source|destination|mailbox`
|
||||
- **Tracking**: Last migrated UID per mailbox
|
||||
- **Resume**: Otomatis lanjut dari UID terakhir yang sukses
|
||||
- **File**: Default `state.db` (bisa diubah dengan `--state`)
|
||||
|
||||
## Logging
|
||||
|
||||
- **Console**: Selalu aktif dengan timestamp
|
||||
- **File**: Optional dengan flag `--log filename.log`
|
||||
- **Format**: `[timestamp] [level] message`
|
||||
- **Levels**: INFO, WARN, ERROR, DEBUG
|
||||
|
||||
## Error Handling
|
||||
|
||||
- **Connection errors**: Retry dengan backoff
|
||||
- **Authentication**: Clear error message
|
||||
- **Mailbox errors**: Skip dan lanjut ke mailbox berikutnya
|
||||
- **Message errors**: Skip message yang corrupt, lanjut ke berikutnya
|
||||
- **Batch errors**: Error di satu akun tidak stop seluruh batch
|
||||
|
||||
## Performance
|
||||
|
||||
- **Batch size**: 50 messages per batch (configurable dalam code)
|
||||
- **Memory**: Efficient streaming untuk message besar
|
||||
- **Concurrent**: Single-threaded untuk stability
|
||||
- **Resume**: Minimal overhead dengan UID tracking
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
**1. Certificate Errors**
|
||||
```
|
||||
x509: certificate signed by unknown authority
|
||||
```
|
||||
**Solution**: Gunakan flag `--insecure`
|
||||
|
||||
**2. Authentication Failed**
|
||||
```
|
||||
login failed: Invalid credentials
|
||||
```
|
||||
**Solution**:
|
||||
- Cek username/password
|
||||
- Untuk Gmail: gunakan App Password, bukan password biasa
|
||||
- Cek apakah IMAP enabled di server
|
||||
|
||||
**3. Connection Timeout**
|
||||
```
|
||||
dial failed: i/o timeout
|
||||
```
|
||||
**Solution**:
|
||||
- Cek hostname dan port
|
||||
- Cek firewall/network connectivity
|
||||
- Cek apakah server support IMAP
|
||||
|
||||
**4. Mailbox Not Found**
|
||||
```
|
||||
select source INBOX: Mailbox doesn't exist
|
||||
```
|
||||
**Solution**:
|
||||
- Cek apakah mailbox ada di source
|
||||
- Beberapa server case-sensitive untuk nama mailbox
|
||||
|
||||
### Debug Mode
|
||||
|
||||
Untuk debug lebih detail, edit code dan set:
|
||||
```go
|
||||
logrus.SetLevel(logrus.DebugLevel)
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
### Gmail to Gmail
|
||||
```bash
|
||||
./mail-migrator --src "olduser@gmail.com:apppass1@imap.gmail.com:993" \
|
||||
--dst "newuser@gmail.com:apppass2@imap.gmail.com:993" \
|
||||
--insecure --resume --log gmail-migration.log
|
||||
```
|
||||
|
||||
### Office365 Migration
|
||||
```bash
|
||||
./mail-migrator --src "user@old.com:password@outlook.office365.com:993" \
|
||||
--dst "user@new.com:password@outlook.office365.com:993" \
|
||||
--resume --log o365-migration.log
|
||||
```
|
||||
|
||||
### Self-hosted with Self-signed Cert
|
||||
```bash
|
||||
./mail-migrator --src "user:pass@mail.old.local:993" \
|
||||
--dst "user:pass@mail.new.local:993" \
|
||||
--insecure --resume
|
||||
```
|
||||
|
||||
## Development
|
||||
|
||||
### Project Structure
|
||||
```
|
||||
├── main.go # CLI interface dan orchestration
|
||||
├── imap.go # IMAP connection dan message copying
|
||||
├── state.go # BoltDB state persistence
|
||||
├── go.mod # Go module dependencies
|
||||
└── README.md # Documentation
|
||||
```
|
||||
|
||||
### Dependencies
|
||||
- `github.com/emersion/go-imap` - IMAP client library
|
||||
- `github.com/urfave/cli/v2` - CLI framework
|
||||
- `go.etcd.io/bbolt` - Embedded key-value database
|
||||
- `github.com/sirupsen/logrus` - Structured logging
|
||||
|
||||
### Building
|
||||
```bash
|
||||
# Development build
|
||||
go run . --help
|
||||
|
||||
# Production build
|
||||
go build -ldflags="-s -w" .
|
||||
|
||||
# Cross-compile for Linux
|
||||
GOOS=linux GOARCH=amd64 go build .
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
MIT License - see LICENSE file for details.
|
||||
|
||||
## Contributing
|
||||
|
||||
1. Fork the repository
|
||||
2. Create feature branch
|
||||
3. Make changes
|
||||
4. Add tests if applicable
|
||||
5. Submit pull request
|
||||
|
||||
## Support
|
||||
|
||||
For issues and questions:
|
||||
1. Check troubleshooting section
|
||||
2. Search existing issues
|
||||
3. Create new issue with:
|
||||
- Command used
|
||||
- Full error message
|
||||
- Log output (with sensitive info removed)
|
||||
4
example-accounts.csv
Normal file
4
example-accounts.csv
Normal file
@@ -0,0 +1,4 @@
|
||||
user1:password1@old-server.com:993,user1:newpassword1@new-server.com:993
|
||||
user2:password2@old-server.com:993,user2:newpassword2@new-server.com:993
|
||||
user3:password3@old-server.com:993,user3:newpassword3@new-server.com:993
|
||||
admin@company.old:adminpass@mail.old.com:993,admin@company.new:adminpass@mail.new.com:993
|
||||
|
10
go.mod
Normal file
10
go.mod
Normal file
@@ -0,0 +1,10 @@
|
||||
module github.com/example/mail-migrator
|
||||
|
||||
go 1.22
|
||||
|
||||
require (
|
||||
github.com/emersion/go-imap v1.2.1
|
||||
github.com/urfave/cli/v2 v2.25.7
|
||||
go.etcd.io/bbolt v1.3.8
|
||||
github.com/sirupsen/logrus v1.9.3
|
||||
)
|
||||
38
go.sum
Normal file
38
go.sum
Normal file
@@ -0,0 +1,38 @@
|
||||
github.com/cpuguy83/go-md2man/v2 v2.0.2 h1:p1EgwI/C7NhT0JmVkwCD2ZBK8j4aeHQX2pMHHBfMQ6w=
|
||||
github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/emersion/go-imap v1.2.1 h1:+s9ZjMEjOB8NzZMVTM3cCenz2JrQIGGo5j1df19WjTA=
|
||||
github.com/emersion/go-imap v1.2.1/go.mod h1:Qlx1FSx2FTxjnjWpIlVNEuX+ylerZQNFE5NsmKFSejY=
|
||||
github.com/emersion/go-message v0.15.0/go.mod h1:wQUEfE+38+7EW8p8aZ96ptg6bAb1iwdgej19uXASlE4=
|
||||
github.com/emersion/go-sasl v0.0.0-20200509203442-7bfe0ed36a21 h1:OJyUGMJTzHTd1XQp98QTaHernxMYzRaOasRir9hUlFQ=
|
||||
github.com/emersion/go-sasl v0.0.0-20200509203442-7bfe0ed36a21/go.mod h1:iL2twTeMvZnrg54ZoPDNfJaJaqy0xIQFuBdrLsmspwQ=
|
||||
github.com/emersion/go-textwrapper v0.0.0-20200911093747-65d896831594/go.mod h1:aqO8z8wPrjkscevZJFVE1wXJrLpC5LtJG7fqLOsPb2U=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk=
|
||||
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
|
||||
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
|
||||
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
|
||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||
github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk=
|
||||
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
|
||||
github.com/urfave/cli/v2 v2.25.7 h1:VAzn5oq403l5pHjc4OhD54+XGO9cdKVL/7lDjF+iKUs=
|
||||
github.com/urfave/cli/v2 v2.25.7/go.mod h1:8qnjx1vcq5s2/wpsqoZFndg2CE5tNFyrTvS6SinrnYQ=
|
||||
github.com/xrash/smetrics v0.0.0-20201216005158-039620a65673 h1:bAn7/zixMGCfxrRTfdpNzjtPYqr8smhKouy9mxVdGPU=
|
||||
github.com/xrash/smetrics v0.0.0-20201216005158-039620a65673/go.mod h1:N3UwUGtsrSj3ccvlPHLoLsHnpR27oXr4ZE984MbSER8=
|
||||
go.etcd.io/bbolt v1.3.8 h1:xs88BrvEv273UsB79e0hcVrlUWmS0a8upikMFhSyAtA=
|
||||
go.etcd.io/bbolt v1.3.8/go.mod h1:N9Mkw9X8x5fupy0IKsmuqVtoGDyxsaDlbk4Rd05IAQw=
|
||||
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.4.0 h1:Zr2JFtRQNX3BCZ8YtxRE9hNJYC8J6I1MVbMg6owUp18=
|
||||
golang.org/x/sys v0.4.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
|
||||
golang.org/x/text v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
|
||||
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
|
||||
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
220
imap.go
Normal file
220
imap.go
Normal file
@@ -0,0 +1,220 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"crypto/tls"
|
||||
"fmt"
|
||||
"net/url"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/emersion/go-imap"
|
||||
"github.com/emersion/go-imap/client"
|
||||
"github.com/sirupsen/logrus"
|
||||
bolt "go.etcd.io/bbolt"
|
||||
)
|
||||
|
||||
// IMAPConfig holds connection details
|
||||
type IMAPConfig struct {
|
||||
Host string
|
||||
Port int
|
||||
Username string
|
||||
Password string
|
||||
UseTLS bool
|
||||
}
|
||||
|
||||
// parseIMAPURL parses "user:pass@host:port" format
|
||||
func parseIMAPURL(rawURL string) (*IMAPConfig, error) {
|
||||
if !strings.Contains(rawURL, "://") {
|
||||
rawURL = "imap://" + rawURL
|
||||
}
|
||||
|
||||
u, err := url.Parse(rawURL)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
port := 993 // default IMAPS
|
||||
if u.Port() != "" {
|
||||
port, err = strconv.Atoi(u.Port())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
password, _ := u.User.Password()
|
||||
|
||||
return &IMAPConfig{
|
||||
Host: u.Hostname(),
|
||||
Port: port,
|
||||
Username: u.User.Username(),
|
||||
Password: password,
|
||||
UseTLS: port == 993 || u.Scheme == "imaps",
|
||||
}, nil
|
||||
}
|
||||
|
||||
// connectIMAP creates IMAP client connection
|
||||
func connectIMAP(cfg *IMAPConfig, insecure bool) (*client.Client, error) {
|
||||
addr := fmt.Sprintf("%s:%d", cfg.Host, cfg.Port)
|
||||
|
||||
var c *client.Client
|
||||
var err error
|
||||
|
||||
if cfg.UseTLS {
|
||||
tlsConfig := &tls.Config{
|
||||
ServerName: cfg.Host,
|
||||
InsecureSkipVerify: insecure,
|
||||
}
|
||||
c, err = client.DialTLS(addr, tlsConfig)
|
||||
} else {
|
||||
c, err = client.Dial(addr)
|
||||
}
|
||||
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("dial failed: %w", err)
|
||||
}
|
||||
|
||||
if err := c.Login(cfg.Username, cfg.Password); err != nil {
|
||||
c.Close()
|
||||
return nil, fmt.Errorf("login failed: %w", err)
|
||||
}
|
||||
|
||||
logrus.Infof("Connected to %s as %s", addr, cfg.Username)
|
||||
return c, nil
|
||||
}
|
||||
|
||||
// listMailboxes returns all mailboxes
|
||||
func listMailboxes(c *client.Client) ([]*imap.MailboxInfo, error) {
|
||||
mailboxes := make(chan *imap.MailboxInfo, 10)
|
||||
done := make(chan error, 1)
|
||||
|
||||
go func() {
|
||||
done <- c.List("", "*", mailboxes)
|
||||
}()
|
||||
|
||||
var result []*imap.MailboxInfo
|
||||
for m := range mailboxes {
|
||||
result = append(result, m)
|
||||
}
|
||||
|
||||
if err := <-done; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func copyMessages(srcClient, dstClient *client.Client, srcMbox, dstMbox string, stateKey string, db *bolt.DB) error {
|
||||
// Select source mailbox
|
||||
srcStatus, err := srcClient.Select(srcMbox, true) // read-only
|
||||
if err != nil {
|
||||
return fmt.Errorf("select source %s: %w", srcMbox, err)
|
||||
}
|
||||
|
||||
// Select destination mailbox
|
||||
_, err = dstClient.Select(dstMbox, false)
|
||||
if err != nil {
|
||||
// Try to create mailbox if it doesn't exist
|
||||
if err := dstClient.Create(dstMbox); err != nil {
|
||||
return fmt.Errorf("create dest %s: %w", dstMbox, err)
|
||||
}
|
||||
if _, err := dstClient.Select(dstMbox, false); err != nil {
|
||||
return fmt.Errorf("select dest %s: %w", dstMbox, err)
|
||||
}
|
||||
}
|
||||
|
||||
// Get last migrated UID for resume
|
||||
var lastUID uint32 = 0
|
||||
if db != nil {
|
||||
lastUID, _ = getLastUID(db, stateKey)
|
||||
}
|
||||
|
||||
logrus.Infof("Migrating %s: %d messages (resume from UID %d)", srcMbox, srcStatus.Messages, lastUID)
|
||||
|
||||
// Search for messages to copy
|
||||
criteria := &imap.SearchCriteria{}
|
||||
if lastUID > 0 {
|
||||
criteria.Uid = &imap.SeqSet{}
|
||||
criteria.Uid.AddRange(lastUID+1, 0) // from lastUID+1 to end
|
||||
} else {
|
||||
criteria.Uid = &imap.SeqSet{}
|
||||
criteria.Uid.AddRange(1, 0) // all messages
|
||||
}
|
||||
|
||||
uids, err := srcClient.UidSearch(criteria)
|
||||
if err != nil {
|
||||
return fmt.Errorf("search failed: %w", err)
|
||||
}
|
||||
|
||||
if len(uids) == 0 {
|
||||
logrus.Infof("No new messages to migrate in %s", srcMbox)
|
||||
return nil
|
||||
}
|
||||
|
||||
logrus.Infof("Found %d messages to migrate", len(uids))
|
||||
|
||||
// Copy messages in batches
|
||||
batchSize := 50
|
||||
for i := 0; i < len(uids); i += batchSize {
|
||||
end := i + batchSize
|
||||
if end > len(uids) {
|
||||
end = len(uids)
|
||||
}
|
||||
|
||||
batch := uids[i:end]
|
||||
seqSet := &imap.SeqSet{}
|
||||
for _, uid := range batch {
|
||||
seqSet.AddNum(uid)
|
||||
}
|
||||
|
||||
// Fetch messages
|
||||
messages := make(chan *imap.Message, batchSize)
|
||||
done := make(chan error, 1)
|
||||
|
||||
go func() {
|
||||
done <- srcClient.UidFetch(seqSet, []imap.FetchItem{imap.FetchRFC822}, messages)
|
||||
}()
|
||||
|
||||
// Process each message
|
||||
for msg := range messages {
|
||||
if len(msg.Body) == 0 {
|
||||
logrus.Warnf("Empty message body for UID %d", msg.Uid)
|
||||
continue
|
||||
}
|
||||
|
||||
// Get message content
|
||||
var msgContent []byte
|
||||
for _, body := range msg.Body {
|
||||
buf := make([]byte, 1024*1024) // 1MB buffer
|
||||
n, err := body.Read(buf)
|
||||
if err != nil && err.Error() != "EOF" {
|
||||
logrus.Errorf("Read message UID %d: %v", msg.Uid, err)
|
||||
continue
|
||||
}
|
||||
msgContent = append(msgContent, buf[:n]...)
|
||||
}
|
||||
|
||||
// Append to destination
|
||||
if err := dstClient.Append(dstMbox, nil, time.Time{}, strings.NewReader(string(msgContent))); err != nil {
|
||||
logrus.Errorf("Append message UID %d: %v", msg.Uid, err)
|
||||
continue
|
||||
}
|
||||
|
||||
// Update state
|
||||
if db != nil {
|
||||
// TODO: implement setLastUID from state.go
|
||||
// setLastUID(db.(*bolt.DB), stateKey, msg.Uid)
|
||||
}
|
||||
|
||||
logrus.Debugf("Copied message UID %d", msg.Uid)
|
||||
}
|
||||
|
||||
if err := <-done; err != nil {
|
||||
return fmt.Errorf("fetch batch failed: %w", err)
|
||||
}
|
||||
|
||||
logrus.Infof("Migrated batch %d-%d/%d", i+1, end, len(uids))
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
BIN
mail-migrator.exe
Normal file
BIN
mail-migrator.exe
Normal file
Binary file not shown.
179
main.go
Normal file
179
main.go
Normal file
@@ -0,0 +1,179 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/csv"
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"os"
|
||||
"time"
|
||||
|
||||
"github.com/sirupsen/logrus"
|
||||
"github.com/urfave/cli/v2"
|
||||
bolt "go.etcd.io/bbolt"
|
||||
)
|
||||
|
||||
func main() {
|
||||
app := &cli.App{
|
||||
Name: "mail-migrator",
|
||||
Usage: "Migrate IMAP mailboxes between servers",
|
||||
Flags: []cli.Flag{
|
||||
&cli.StringFlag{Name: "src", Usage: "Source IMAP URL (user:pass@host:port)"},
|
||||
&cli.StringFlag{Name: "dst", Usage: "Destination IMAP URL (user:pass@host:port)"},
|
||||
&cli.StringFlag{Name: "batch", Usage: "CSV file with src,dst entries for batch migration"},
|
||||
&cli.BoolFlag{Name: "insecure", Usage: "Allow insecure TLS (trust self-signed certs)"},
|
||||
&cli.BoolFlag{Name: "resume", Usage: "Resume incomplete migrations"},
|
||||
&cli.StringFlag{Name: "state", Value: "state.db", Usage: "Path to state DB (Bolt)"},
|
||||
&cli.StringFlag{Name: "log", Usage: "Path to log file (optional)"},
|
||||
},
|
||||
Action: func(c *cli.Context) error {
|
||||
setupLogging(c.String("log"))
|
||||
insecure := c.Bool("insecure")
|
||||
resume := c.Bool("resume")
|
||||
statePath := c.String("state")
|
||||
|
||||
if batch := c.String("batch"); batch != "" {
|
||||
return runBatch(batch, insecure, resume, statePath)
|
||||
}
|
||||
|
||||
src := c.String("src")
|
||||
dst := c.String("dst")
|
||||
if src == "" || dst == "" {
|
||||
return fmt.Errorf("either src/dst or batch must be provided")
|
||||
}
|
||||
return migrateOne(context.Background(), src, dst, insecure, resume, statePath)
|
||||
},
|
||||
}
|
||||
|
||||
if err := app.Run(os.Args); err != nil {
|
||||
logrus.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func setupLogging(file string) {
|
||||
logrus.SetFormatter(&logrus.TextFormatter{FullTimestamp: true})
|
||||
logrus.SetLevel(logrus.InfoLevel)
|
||||
if file != "" {
|
||||
f, err := os.OpenFile(file, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0644)
|
||||
if err != nil {
|
||||
log.Fatalf("unable to open log file: %v", err)
|
||||
}
|
||||
logrus.SetOutput(io.MultiWriter(os.Stdout, f))
|
||||
}
|
||||
}
|
||||
|
||||
func runBatch(path string, insecure, resume bool, statePath string) error {
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer f.Close()
|
||||
reader := csv.NewReader(f)
|
||||
for {
|
||||
rec, err := reader.Read()
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if len(rec) < 2 {
|
||||
logrus.Warn("invalid record, skipping")
|
||||
continue
|
||||
}
|
||||
src, dst := rec[0], rec[1]
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Minute)
|
||||
if err := migrateOne(ctx, src, dst, insecure, resume, statePath); err != nil {
|
||||
logrus.Errorf("migration failed for %s -> %s: %v", src, dst, err)
|
||||
}
|
||||
cancel()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// migrateOne performs migration for single account
|
||||
func migrateOne(ctx context.Context, src, dst string, insecure, resume bool, statePath string) error {
|
||||
logrus.Infof("Starting migration %s -> %s (insecure=%v, resume=%v)", src, dst, insecure, resume)
|
||||
|
||||
// Parse IMAP URLs
|
||||
srcCfg, err := parseIMAPURL(src)
|
||||
if err != nil {
|
||||
return fmt.Errorf("parse source URL: %w", err)
|
||||
}
|
||||
|
||||
dstCfg, err := parseIMAPURL(dst)
|
||||
if err != nil {
|
||||
return fmt.Errorf("parse destination URL: %w", err)
|
||||
}
|
||||
|
||||
// Open state database if resume is enabled
|
||||
var db *bolt.DB
|
||||
if resume {
|
||||
db, err = openStateDB(statePath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("open state DB: %w", err)
|
||||
}
|
||||
defer db.Close()
|
||||
}
|
||||
|
||||
// Connect to source
|
||||
srcClient, err := connectIMAP(srcCfg, insecure)
|
||||
if err != nil {
|
||||
return fmt.Errorf("connect to source: %w", err)
|
||||
}
|
||||
defer srcClient.Logout()
|
||||
|
||||
// Connect to destination
|
||||
dstClient, err := connectIMAP(dstCfg, insecure)
|
||||
if err != nil {
|
||||
return fmt.Errorf("connect to destination: %w", err)
|
||||
}
|
||||
defer dstClient.Close()
|
||||
|
||||
// List mailboxes from source
|
||||
mailboxes, err := listMailboxes(srcClient)
|
||||
if err != nil {
|
||||
return fmt.Errorf("list mailboxes: %w", err)
|
||||
}
|
||||
|
||||
logrus.Infof("Found %d mailboxes to migrate", len(mailboxes))
|
||||
|
||||
// Migrate each mailbox
|
||||
for _, mbox := range mailboxes {
|
||||
// Skip certain system mailboxes
|
||||
if shouldSkipMailbox(mbox.Name) {
|
||||
logrus.Debugf("Skipping mailbox: %s", mbox.Name)
|
||||
continue
|
||||
}
|
||||
|
||||
stateKey := formatStateKey(src, dst, mbox.Name)
|
||||
|
||||
logrus.Infof("Migrating mailbox: %s", mbox.Name)
|
||||
if err := copyMessages(srcClient, dstClient, mbox.Name, mbox.Name, stateKey, db); err != nil {
|
||||
logrus.Errorf("Failed to migrate mailbox %s: %v", mbox.Name, err)
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
logrus.Infof("Migration completed successfully")
|
||||
return nil
|
||||
}
|
||||
|
||||
// shouldSkipMailbox determines if a mailbox should be skipped
|
||||
func shouldSkipMailbox(name string) bool {
|
||||
// Skip common system/virtual mailboxes
|
||||
skipList := []string{
|
||||
"[Gmail]/All Mail",
|
||||
"[Gmail]/Important",
|
||||
"[Gmail]/Starred",
|
||||
"[Gmail]/Chats",
|
||||
}
|
||||
|
||||
for _, skip := range skipList {
|
||||
if name == skip {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
51
state.go
Normal file
51
state.go
Normal file
@@ -0,0 +1,51 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/binary"
|
||||
"fmt"
|
||||
|
||||
bolt "go.etcd.io/bbolt"
|
||||
)
|
||||
|
||||
var bucketName = []byte("migrator_state")
|
||||
|
||||
// openStateDB opens (or creates) a BoltDB at given path.
|
||||
func openStateDB(path string) (*bolt.DB, error) {
|
||||
return bolt.Open(path, 0600, nil)
|
||||
}
|
||||
|
||||
// getLastUID returns last migrated UID for given key.
|
||||
func getLastUID(db *bolt.DB, key string) (uint32, error) {
|
||||
var uid uint32
|
||||
err := db.View(func(tx *bolt.Tx) error {
|
||||
b := tx.Bucket(bucketName)
|
||||
if b == nil {
|
||||
return nil
|
||||
}
|
||||
v := b.Get([]byte(key))
|
||||
if v == nil {
|
||||
return nil
|
||||
}
|
||||
uid = binary.BigEndian.Uint32(v)
|
||||
return nil
|
||||
})
|
||||
return uid, err
|
||||
}
|
||||
|
||||
// setLastUID updates last migrated UID for given key.
|
||||
func setLastUID(db *bolt.DB, key string, uid uint32) error {
|
||||
return db.Update(func(tx *bolt.Tx) error {
|
||||
b, err := tx.CreateBucketIfNotExists(bucketName)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
buf := make([]byte, 4)
|
||||
binary.BigEndian.PutUint32(buf, uid)
|
||||
return b.Put([]byte(key), buf)
|
||||
})
|
||||
}
|
||||
|
||||
// formatStateKey creates state key from src, dst and mailbox.
|
||||
func formatStateKey(src, dst, mbox string) string {
|
||||
return fmt.Sprintf("%s|%s|%s", src, dst, mbox)
|
||||
}
|
||||
Reference in New Issue
Block a user