dualfroz/AfterShip-email-verifier
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Upstream AfterShip/email-verifier state of the touched files before 072e335125

Browse Source
This commit is contained in:
upstream committed 2026-09-11 23:05:39 +02:00
commit 74c6cde98a
4 files changed
+1273
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

README.md
+352
View File
@@ -0,0 +1,352 @@
# email-verifier
✉️ A Go library for email verification without sending any emails.
[![Build Status](https://github.com/AfterShip/email-verifier/workflows/CI%20Actions/badge.svg)](https://github.com/AfterShip/email-verifier/actions)
[![Godoc](http://img.shields.io/badge/godoc-reference-blue.svg?style=flat)](https://godoc.org/github.com/AfterShip/email-verifier)
[![Coverage Status](https://coveralls.io/repos/github/AfterShip/email-verifier/badge.svg?t=VTgVfL)](https://coveralls.io/github/AfterShip/email-verifier)
[![Go Report Card](https://goreportcard.com/badge/github.com/AfterShip/email-verifier)](https://goreportcard.com/report/github.com/AfterShip/email-verifier)
[![license](http://img.shields.io/badge/license-MIT-red.svg?style=flat)](https://github.com/AfterShip/email-verifier/blob/main/LICENSE)
## Features
- Email Address Validation: validates if a string contains a valid email.
- Email Verification Lookup via SMTP: performs an email verification on the passed email (catchAll detection enabled by default)
- MX Validation: checks the DNS MX records for the given domain name
- Misc Validation: including Free email provider check, Role account validation, Disposable emails address (DEA) validation
- Email Reachability: checks how confident in sending an email to the address
## Install
Use `go get` to install this package.
```shell script
go get -u github.com/AfterShip/email-verifier
```
### Supported Go versions
The three most recent Go releases, currently **1.25, 1.26 and 1.27**, each covered by CI.
`go.mod` requires the oldest of them, so an older toolchain will refuse to build this module.
## Usage
### Basic usage
Use `Verify` method to verify an email address with different dimensions
```go
package main
import (
"fmt"
emailverifier "github.com/AfterShip/email-verifier"
)
var (
verifier = emailverifier.NewVerifier()
)
func main() {
email := "example@exampledomain.org"
ret, err := verifier.Verify(email)
if err != nil {
fmt.Println("verify email address failed, error is: ", err)
return
}
if !ret.Syntax.Valid {
fmt.Println("email address syntax is invalid")
return
}
fmt.Println("email validation result", ret)
/*
result is:
{
"email":"example@exampledomain.org",
"disposable":false,
"reachable":"unknown",
"role_account":false,
"free":false,
"syntax":{
"username":"example",
"domain":"exampledomain.org",
"valid":true
},
"has_mx_records":true,
"smtp":null,
"gravatar":null
}
*/
}
```
### Email verification Lookup
Use `CheckSMTP` to performs an email verification lookup via SMTP.
```go
var (
verifier = emailverifier.
NewVerifier().
EnableSMTPCheck()
)
func main() {
domain := "domain.org"
username := "username"
ret, err := verifier.CheckSMTP(domain, username)
if err != nil {
fmt.Println("check smtp failed: ", err)
return
}
fmt.Println("smtp validation result: ", ret)
}
```
If you want to disable catchAll checking, use the `DisableCatchAllCheck()` switch (in effect only when SMTP verification is enabled).
```go
verifier = emailverifier.
NewVerifier().
EnableSMTPCheck().
DisableCatchAllCheck()
```
> Note: because most of the ISPs block outgoing SMTP requests through port 25 to prevent email spamming, the module will not perform SMTP checking by default. You can initialize the verifier with `EnableSMTPCheck()` to enable such capability if port 25 is usable,
> or use a socks proxy to connect over SMTP
> Note: set `FromEmail()` and `HelloName()` before verifying at any volume. The defaults are
> `user@example.org` and `localhost`, and `example.org` is reserved by RFC 2606, so a server
> that validates the sender can reject the whole exchange before it ever considers the
> address you asked about. Rejections of this kind arrive at `MAIL FROM` and name the sender
> or the connecting host rather than the recipient, for example
> `550 5.7.25 Forward-confirmed reverse DNS failed` or
> `550 5.7.1 Service unavailable, Client host [...] blocked using Spamhaus`. Use a domain you
> control, with a PTR record for the IP you connect from.
### Use a SOCKS5 proxy to verify email
Support setting a SOCKS5 proxy to verify the email, proxyURI should be in the format: `socks5://user:password@127.0.0.1:1080`
The protocol could be socks5, socks4 and socks4a.
```go
var (
verifier = emailverifier.
NewVerifier().
EnableSMTPCheck().
Proxy("socks5://user:password@127.0.0.1:1080")
)
func main() {
domain := "domain.org"
username := "username"
ret, err := verifier.CheckSMTP(domain, username)
if err != nil {
fmt.Println("check smtp failed: ", err)
return
}
fmt.Println("smtp validation result: ", ret)
}
```
Two things to know about the proxy:
> **A query string in the proxy URI is ignored.** `golang.org/x/net/proxy` reads only the
> scheme, credentials and host, so a `?timeout=5s` has no effect. Use `ConnectTimeout()` and
> `OperationTimeout()` instead.
> **DNS does not go through the proxy.** Only the connection to the mail server does, so MX
> lookups still leave from the local machine. To route them through the proxy as well, dial
> your DNS server over TCP through it — SOCKS5 carries TCP, and Go frames DNS over a
> non-packet connection per RFC 7766:
>
> ```go
> socksDialer, err := proxy.SOCKS5("tcp", "127.0.0.1:1080", nil, proxy.Direct)
> if err != nil {
> return err
> }
> verifier = emailverifier.
> NewVerifier().
> EnableSMTPCheck().
> Proxy("socks5://127.0.0.1:1080").
> Resolver(&net.Resolver{
> PreferGo: true,
> Dial: func(ctx context.Context, network, address string) (net.Conn, error) {
> // ignore network, which may be "udp": SOCKS5 CONNECT carries TCP only
> return socksDialer.Dial("tcp", "8.8.8.8:53")
> },
> })
> ```
>
> Each lookup opens a TCP connection through the proxy, so this costs more than plain UDP DNS.
### Use a custom DNS resolver
By default, the verifier uses `net.DefaultResolver` (the platform's system-configured DNS resolver) for MX lookups and direct SMTP host lookups. You can override this behaviour by supplying your own `*net.Resolver` via `Resolver()`, for example to query a specific DNS server. When a proxy is configured, SMTP hostname resolution follows the proxy path rather than this resolver. MX record lookups always use this resolver, whether or not a proxy is set.
```go
var (
verifier = emailverifier.
NewVerifier().
EnableSMTPCheck().
Resolver(&net.Resolver{
PreferGo: true,
Dial: func(ctx context.Context, network, address string) (net.Conn, error) {
d := net.Dialer{}
return d.DialContext(ctx, network, "8.8.8.8:53")
},
})
)
func main() {
domain := "domain.org"
username := "username"
ret, err := verifier.CheckSMTP(domain, username)
if err != nil {
fmt.Println("check smtp failed: ", err)
return
}
fmt.Println("smtp validation result: ", ret)
}
```
> Note: `PreferGo: true` is doing real work here, it is not boilerplate. MX record lookups always
> go through Go's own DNS client, so a custom `Dial` is honoured for those either way. Resolving
> the mail server's hostname to an IP for the TCP connection is different: without `PreferGo` that
> step may use the system (cgo) resolver, which ignores `Dial` entirely and quietly falls back to
> `/etc/resolv.conf`. Even with it set, `/etc/hosts` is still consulted before DNS, so a host
> listed there never reaches your resolver.
### Misc Validation
To check if an email domain is disposable via `IsDisposable`
```go
var (
verifier = emailverifier.
NewVerifier().
EnableAutoUpdateDisposable()
)
func main() {
domain := "domain.org"
if verifier.IsDisposable(domain) {
fmt.Printf("%s is a disposable domain\n", domain)
return
}
fmt.Printf("%s is not a disposable domain\n", domain)
}
```
> Note: It is possible to automatically update the disposable domains daily by initializing verifier with `EnableAutoUpdateDisposable()`
### Suggestions for domain typo
Will check for typos in an email domain in addition to evaluating its validity.
If we detect a possible typo, you will find a non-empty "suggestion" field in the validation result containing what we believe to be the correct domain.
Also, you can use the `SuggestDomain()` method alone to check the domain for possible misspellings
```go
func main() {
domain := "gmai.com"
suggestion := verifier.SuggestDomain(domain)
// suggestion should be `gmail.com`
if suggestion != "" {
fmt.Printf("domain %s is misspelled, right domain is %s. \n", domain, suggestion)
return
}
fmt.Printf("domain %s has no possible misspellings. \n", domain)
}
```
> Note: When using the `Verify()` method, domain typo checking is not enabled by default, you can enable it in a verifier with `EnableDomainSuggest()`
For more detailed documentation, please check on godoc.org 👉 [email-verifier](https://godoc.org/github.com/AfterShip/email-verifier)
## API
We provide a simple **self-hosted** [API server](https://github.com/AfterShip/email-verifier/tree/main/cmd/apiserver) script for reference.
The API interface is very simple. All you need to do is to send a GET request with the following URL.
The `email` parameter would be the target email you want to verify.
`https://{your_host}/v1/{email}/verification`
## Similar Libraries Comparison
| | [email-verifier](https://github.com/AfterShip/email-verifier) | [trumail](https://github.com/trumail/trumail) | [check-if-email-exists](https://reacher.email/) | [freemail](https://github.com/willwhite/freemail) |
| ----------------------------------- | :----------------------------------------------------------: | :-------------------------------------------: | :---------------------------------------------: | :-----------------------------------------------: |
| **Features** | 〰️ | 〰️ | 〰️ | 〰️ |
| Disposable email address validation | ✅ | ✅, but not available in free lib | ✅ | ✅ |
| Disposable address autoupdate | ✅ | 🤔 | ❌ | ❌ |
| Free email provider check | ✅ | ✅, but not available in free lib | ❌ | ✅ |
| Role account validation | ✅ | ❌ | ✅ | ❌ |
| Syntax validation | ✅ | ✅ | ✅ | ❌ |
| Email reachability | ✅ | ✅ | ✅ | ❌ |
| DNS records validation | ✅ | ✅ | ✅ | ❌ |
| Email deliverability | ✅ | ✅ | ✅ | ❌ |
| Mailbox disabled | ✅ | ✅ | ✅ | ❌ |
| Full inbox | ✅ | ✅ | ✅ | ❌ |
| Host exists | ✅ | ✅ | ✅ | ❌ |
| Catch-all | ✅ | ✅ | ✅ | ❌ |
| Gravatar | ✅ | ✅, but not available in free lib | ❌ | ❌ |
| Typo check | ✅ | ✅, but not available in free lib | ❌ | ❌ |
| Use proxy to connect over SMTP | ✅ | ❌ | ✅ | ❌ |
| Honeyport dection | 🔜 | ❌ | ❌ | ❌ |
| Bounce email check | 🔜 | ❌ | ❌ | ❌ |
| **Tech** | 〰️ | 〰️ | 〰️ | 〰️ |
| Provide API | ✅ | ✅ | ✅ | ❌ |
| Free API | ✅ | ❌ | ❌ | ❌ |
| Language | Go | Go | Rust | JavaScript |
| Active maintain | ✅ | ❌ | ✅ | ✅ |
| High Performance | ✅ | ❌ | ✅ | ✅ |
## FAQ
#### The library hangs/takes a long time after 30 seconds when performing email verification lookup via SMTP
Most ISPs block outgoing SMTP requests through port 25 to prevent email spamming. `email-verifier` needs to have this port open to make a connection to the email's SMTP server. With the port being blocked, it is not possible to perform such checking, and it will instead hang until timeout error. Unfortunately, there is no easy workaround for this issue.
For more information, you may also visit [this StackOverflow thread](https://stackoverflow.com/questions/18139102/how-to-get-around-an-isp-block-on-port-25-for-smtp).
#### The output shows `"connection refused"` in the `smtp.error` field.
This error can also be due to SMTP ports being blocked by the ISP, see the above answer.
#### What does reachable: "unknown" means
This means that the server does not allow real-time verification of an email right now, or the email provider is a catch-all email server.
## Credits
- [trumail](https://github.com/trumail/trumail)
- [check-if-email-exists](https://github.com/amaurymartiny/check-if-email-exists)
- [mailcheck](https://github.com/mailcheck/mailcheck)
- disposable domains from [disposable/disposable-email-domains](https://github.com/disposable/disposable-email-domains), both for the generated list and for `EnableAutoUpdateDisposable()`, less the few real providers it misclassifies (see `cmd/build_metadata/disposable_allowlist.txt`)
- free provider data from [HubSpot's free email domain list](https://f.hubspotusercontent40.net/hubfs/2832391/Marketing/Lead-Capture/free-domains-2.csv) and [tbrianjones/free_email_provider_domains](https://gist.github.com/tbrianjones/5992856), plus carrier and portal mailboxes vendored from [Kikobeats/free-email-domains](https://github.com/Kikobeats/free-email-domains)
## Contributing
For details on contributing to this repository, see the [contributing guide](https://github.com/AfterShip/email-verifier/blob/main/CONTRIBUTING.md).
## License
This package is licensed under MIT license. See [LICENSE](https://github.com/AfterShip/email-verifier/blob/main/LICENSE) for details.
smtp.go
+330
View File
@@ -0,0 +1,330 @@
package emailverifier
import (
"context"
"errors"
"fmt"
"math/rand"
"net"
"net/smtp"
"net/textproto"
"net/url"
"strconv"
"strings"
"sync"
"time"
"golang.org/x/net/proxy"
)
// SMTP stores all information for SMTP verification lookup
type SMTP struct {
HostExists bool `json:"host_exists"` // is the host exists?
FullInbox bool `json:"full_inbox"` // is the email account's inbox full?
CatchAll bool `json:"catch_all"` // does the domain have a catch-all email address?
Deliverable bool `json:"deliverable"` // can send an email to the email server?
Disabled bool `json:"disabled"` // is the email blocked or disabled by the provider?
}
// catchAllOutcome records what the catch-all probe established. SMTP.CatchAll
// cannot: it is also true when the probe never ran, and when the server replied
// about itself rather than about the address.
type catchAllOutcome int
const (
// catchAllNotRun means the probe was disabled, or the exchange ended before it.
catchAllNotRun catchAllOutcome = iota
// catchAllAccepted means a random address was accepted, so the domain takes anything.
catchAllAccepted
// catchAllRefused means a random address was refused as non-existent, so the
// domain does not take just anything.
catchAllRefused
// catchAllInconclusive means the server replied, but about itself rather than
// about the address: a rate limit, a block, a temporary failure.
catchAllInconclusive
)
// catchAllFromResult reads an API verifier's flat result as a probe outcome.
// The verifier reports a catch-all as a plain bool with none of our probing
// behind it, so take it at face value: leaving the outcome at catchAllNotRun
// would make a false CatchAll read as a refusal of the address itself.
func catchAllFromResult(s *SMTP) catchAllOutcome {
if s != nil && s.CatchAll {
return catchAllAccepted
}
return catchAllRefused
}
// CheckSMTP performs an email verification on the passed domain via SMTP
// - the domain is the passed email domain
// - username is used to check the deliverability of specific email address,
//
// if server is catch-all server, username will not be checked
func (v *Verifier) CheckSMTP(domain, username string) (*SMTP, error) {
res, _, err := v.checkSMTP(domain, username)
return res, err
}
// checkSMTP is CheckSMTP plus what the catch-all probe concluded. Verify needs
// that to tell a domain which accepts every address from one we could not ask.
func (v *Verifier) checkSMTP(domain, username string) (*SMTP, catchAllOutcome, error) {
if !v.smtpCheckEnabled {
return nil, catchAllNotRun, nil
}
var ret SMTP
var err error
catchAll := catchAllNotRun
email := fmt.Sprintf("%s@%s", username, domain)
// Dial any SMTP server that will accept a connection
client, mx, err := newSMTPClient(domain, v.proxyURI, v.dnsResolver(), v.connectTimeout, v.operationTimeout)
if err != nil {
return &ret, catchAll, ParseSMTPError(err)
}
// Defer quit the SMTP connection
defer client.Close()
// We opened a connection and read the server's greeting, so the host is
// there whatever happens from here on. Recorded before EHLO and MAIL FROM
// so that a rejection at either -- which is about our sender, not about the
// host -- does not report the host as missing.
ret.HostExists = true
// Check by api when enabled and host recognized.
for _, apiVerifier := range v.apiVerifiers {
if apiVerifier.isSupported(strings.ToLower(mx.Host)) {
res, err := apiVerifier.check(domain, username)
return res, catchAllFromResult(res), err
}
}
// Sets the HELO/EHLO hostname
if err = client.Hello(v.helloName); err != nil {
return &ret, catchAll, ParseSMTPError(err)
}
// Sets the from email
if err = client.Mail(v.fromEmail); err != nil {
return &ret, catchAll, ParseSMTPError(err)
}
// Default sets catch-all to true
ret.CatchAll = true
if v.catchAllCheckEnabled {
// Checks the deliver ability of a randomly generated address in
// order to verify the existence of a catch-all and etc.
randomEmail := GenerateRandomEmail(domain)
if err = client.Rcpt(randomEmail); err != nil {
// The server said something, but not necessarily about the address.
// Only a refusal of the address itself rules a catch-all out.
catchAll = catchAllInconclusive
if e := ParseSMTPError(err); e != nil {
switch e.Message {
case ErrFullInbox:
ret.FullInbox = true
case ErrNotAllowed:
ret.Disabled = true
// If The client typically receives a `550 5.1.1` code as a reply to RCPT TO command,
// In most cases, this is because the recipient address does not exist.
case ErrServerUnavailable:
ret.CatchAll = false
catchAll = catchAllRefused
default:
}
}
} else {
catchAll = catchAllAccepted
}
// If the email server is a catch-all email server,
// no need to calibrate deliverable on a specific user
if ret.CatchAll {
return &ret, catchAll, nil
}
}
// If no username provided,
// no need to calibrate deliverable on a specific user
if username == "" {
return &ret, catchAll, nil
}
if err = client.Rcpt(email); err == nil {
ret.Deliverable = true
}
return &ret, catchAll, nil
}
// newSMTPClient generates a new available SMTP client
func newSMTPClient(domain, proxyURI string, resolver *net.Resolver, connectTimeout, operationTimeout time.Duration) (*smtp.Client, *net.MX, error) {
domain = domainToASCII(domain)
mxRecords, err := resolver.LookupMX(context.Background(), domain)
if err != nil {
return nil, nil, err
}
if len(mxRecords) == 0 {
// Capitalised on purpose: ParseSMTPError passes this string through
// verbatim as LookupError.Message, which is user-facing and matches the
// Err* constants in error.go.
return nil, nil, errors.New("No MX records found") //nolint:staticcheck // ST1005
}
// Create a channel for receiving response from
ch := make(chan interface{}, 1)
selectedMXCh := make(chan *net.MX, 1)
// Done indicates if we're still waiting on dial responses
var done bool
// mutex for data race
var mutex sync.Mutex
// Attempt to connect to all SMTP servers concurrently
for i, r := range mxRecords {
addr := r.Host + smtpPort
index := i
go func() {
c, err := dialSMTP(addr, proxyURI, resolver, connectTimeout, operationTimeout)
if err != nil {
if !done {
ch <- err
}
return
}
// Place the client on the channel or close it
mutex.Lock()
switch {
case !done:
done = true
ch <- c
selectedMXCh <- mxRecords[index]
default:
c.Close()
}
mutex.Unlock()
}()
}
// Collect errors or return a client
var errs []error
for {
res := <-ch
switch r := res.(type) {
case *smtp.Client:
return r, <-selectedMXCh, nil
case error:
errs = append(errs, r)
if len(errs) == len(mxRecords) {
return nil, nil, preferredDialError(errs)
}
default:
// Capitalised on purpose; see the note on "No MX records found" above.
return nil, nil, errors.New("Unexpected response dialing SMTP server") //nolint:staticcheck // ST1005
}
}
}
// preferredDialError picks the most useful of the errors gathered while racing
// the MX hosts.
//
// The hosts are dialled concurrently, so the first error to arrive is the one
// that failed fastest -- typically a refused connection or a DNS miss. An error
// carrying an SMTP reply took longer precisely because a server answered and
// said why it was refusing us, which is what the caller needs. Returning the
// first arrival systematically preferred the least informative one.
func preferredDialError(errs []error) error {
for _, err := range errs {
if hasSMTPReply(err) {
return err
}
}
return errs[0]
}
// hasSMTPReply reports whether err carries a reply from an SMTP server, as
// opposed to a network level failure.
func hasSMTPReply(err error) bool {
var protoErr *textproto.Error
if errors.As(err, &protoErr) {
return true
}
// Errors that have already been rendered to a string keep the reply code
// at the front, so fall back to reading it from there.
s := err.Error()
if len(s) < 3 {
return false
}
code, convErr := strconv.Atoi(s[:3])
return convErr == nil && code >= 200 && code < 600
}
// dialSMTP is a timeout wrapper for smtp.Dial. It attempts to dial an
// SMTP server (socks5 proxy supported) and fails with a timeout if timeout is reached while
// attempting to establish a new connection
func dialSMTP(addr, proxyURI string, resolver *net.Resolver, connectTimeout, operationTimeout time.Duration) (*smtp.Client, error) {
// Dial the new smtp connection
var conn net.Conn
var err error
if proxyURI != "" {
conn, err = establishProxyConnection(addr, proxyURI, connectTimeout)
} else {
conn, err = establishConnection(addr, resolver, connectTimeout)
}
if err != nil {
return nil, err
}
// Set specific timeouts for writing and reading
err = conn.SetDeadline(time.Now().Add(operationTimeout))
if err != nil {
return nil, err
}
host, _, _ := net.SplitHostPort(addr)
return smtp.NewClient(conn, host)
}
// GenerateRandomEmail generates a random email address using the domain passed. Used
// primarily for checking the existence of a catch-all address
func GenerateRandomEmail(domain string) string {
r := make([]byte, 32)
for i := range 32 {
r[i] = alphanumeric[rand.Intn(len(alphanumeric))] //nolint:gosec
}
return fmt.Sprintf("%s@%s", string(r), domain)
}
// establishConnection connects to the address on the named network address.
func establishConnection(addr string, resolver *net.Resolver, timeout time.Duration) (net.Conn, error) {
dialer := net.Dialer{Timeout: timeout, Resolver: resolver}
return dialer.Dial("tcp", addr)
}
// establishProxyConnection connects to the address on the named network address
// via proxy protocol
func establishProxyConnection(addr, proxyURI string, timeout time.Duration) (net.Conn, error) {
u, err := url.Parse(proxyURI)
if err != nil {
return nil, err
}
dialer, err := proxy.FromURL(u, nil)
if err != nil {
return nil, err
}
// https://github.com/golang/go/issues/37549#issuecomment-1178745487
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
return dialer.(proxy.ContextDialer).DialContext(ctx, "tcp", addr)
}
smtp_test.go
+291
View File
@@ -0,0 +1,291 @@
package emailverifier
import (
"context"
"errors"
"net"
"net/textproto"
"sync/atomic"
"syscall"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestCheckSMTPUnSupportedVendor(t *testing.T) {
err := verifier.EnableAPIVerifier("unsupported_vendor")
assert.Error(t, err)
}
func TestCheckSMTPOK_HostExists(t *testing.T) {
domain := "github.com"
smtp, err := verifier.CheckSMTP(domain, "")
expected := SMTP{
HostExists: true,
FullInbox: false,
CatchAll: true,
Disabled: false,
}
require.NoError(t, err)
assert.Equal(t, &expected, smtp)
}
func TestCheckSMTPOK_CatchAllHost(t *testing.T) {
domain := "gmail.com"
smtp, err := verifier.CheckSMTP(domain, "")
expected := SMTP{
HostExists: true,
FullInbox: false,
CatchAll: false,
Disabled: false,
}
require.NoError(t, err)
assert.Equal(t, &expected, smtp)
}
func TestCheckSMTPOK_NoCatchAllHost(t *testing.T) {
domain := "gmail.com"
smtp, err := verifier.CheckSMTP(domain, "")
expected := SMTP{
HostExists: true,
FullInbox: false,
CatchAll: false,
Disabled: false,
}
require.NoError(t, err)
assert.Equal(t, &expected, smtp)
}
func TestCheckSMTPOK_NoCatchAllHostCatchAllCheckDisabled(t *testing.T) {
domain := "gmail.com"
var verifier = NewVerifier().EnableSMTPCheck().DisableCatchAllCheck()
smtp, err := verifier.CheckSMTP(domain, "")
expected := SMTP{
HostExists: true,
FullInbox: false,
CatchAll: true,
Disabled: false,
}
require.NoError(t, err)
assert.Equal(t, &expected, smtp)
}
func TestCheckSMTPOK_UpdateFromEmail(t *testing.T) {
domain := "github.com"
verifier.FromEmail("from@email.top")
smtp, err := verifier.CheckSMTP(domain, "")
expected := SMTP{
HostExists: true,
FullInbox: false,
CatchAll: true,
Deliverable: false,
Disabled: false,
}
require.NoError(t, err)
assert.Equal(t, &expected, smtp)
}
func TestCheckSMTPOK_UpdateHelloName(t *testing.T) {
domain := "github.com"
verifier.HelloName("email.top")
smtp, err := verifier.CheckSMTP(domain, "")
expected := SMTP{
HostExists: true,
FullInbox: false,
CatchAll: true,
Deliverable: false,
Disabled: false,
}
require.NoError(t, err)
assert.Equal(t, &expected, smtp)
}
func TestCheckSMTPOK_WithNoExistUsername(t *testing.T) {
domain := "github.com"
username := "testing"
smtp, err := verifier.CheckSMTP(domain, username)
expected := SMTP{
HostExists: true,
FullInbox: false,
CatchAll: true,
Disabled: false,
}
require.NoError(t, err)
assert.Equal(t, &expected, smtp)
}
func TestCheckSMTP_DisabledSMTPCheck(t *testing.T) {
domain := "github.com"
verifier.DisableSMTPCheck()
smtp, err := verifier.CheckSMTP(domain, "username")
verifier.EnableSMTPCheck()
require.NoError(t, err)
assert.Nil(t, smtp)
}
func TestCheckSMTPOK_HostNotExists(t *testing.T) {
domain := "notExistHost.com"
smtp, err := verifier.CheckSMTP(domain, "")
require.ErrorContains(t, err, ErrNoSuchHost)
assert.Equal(t, &SMTP{}, smtp)
}
func TestNewSMTPClientOK(t *testing.T) {
domain := "gmail.com"
timeout := 5 * time.Second
ret, _, err := newSMTPClient(domain, "", net.DefaultResolver, timeout, timeout)
assert.NotNil(t, ret)
assert.NoError(t, err)
}
func TestNewSMTPClientFailed_WithInvalidProxy(t *testing.T) {
domain := "gmail.com"
proxyURI := "socks5://user:password@127.0.0.1:1080?timeout=5s"
timeout := 5 * time.Second
ret, _, err := newSMTPClient(domain, proxyURI, net.DefaultResolver, timeout, timeout)
assert.Nil(t, ret)
require.ErrorIs(t, err, syscall.ECONNREFUSED)
}
func TestNewSMTPClientFailed(t *testing.T) {
domain := "zzzz171777.com"
timeout := 5 * time.Second
ret, _, err := newSMTPClient(domain, "", net.DefaultResolver, timeout, timeout)
assert.Nil(t, ret)
assert.Error(t, err)
}
func TestDialSMTPFailed_NoPortIsConfigured(t *testing.T) {
disposableDomain := "zzzz1717.com"
timeout := 5 * time.Second
ret, err := dialSMTP(disposableDomain, "", net.DefaultResolver, timeout, timeout)
assert.Nil(t, ret)
require.Error(t, err)
assert.Contains(t, err.Error(), "missing port")
}
func TestDialSMTPFailed_NoSuchHost(t *testing.T) {
disposableDomain := "zzzzyyyyaaa123.com:25"
timeout := 5 * time.Second
ret, err := dialSMTP(disposableDomain, "", net.DefaultResolver, timeout, timeout)
assert.Nil(t, ret)
require.Error(t, err)
assert.Contains(t, err.Error(), "no such host")
}
func TestDialSMTP_WithCustomResolver(t *testing.T) {
domain := "github.com:25"
timeout := 5 * time.Second
wantErr := errors.New("custom resolver dial invoked")
var called atomic.Bool
customResolver := &net.Resolver{
PreferGo: true,
Dial: func(ctx context.Context, network, address string) (net.Conn, error) {
called.Store(true)
return nil, wantErr
},
}
ret, err := dialSMTP(domain, "", customResolver, timeout, timeout)
assert.Nil(t, ret)
assert.True(t, called.Load())
require.ErrorContains(t, err, wantErr.Error())
}
// The counterpart to the test above: SOCKS5 hands the target hostname to the
// proxy to resolve remotely, so the custom resolver must not be consulted on
// the proxy path. This pins the behaviour the README documents; if the proxy
// path ever gains resolver support, this test should fail and prompt a docs
// update. The proxy address is an IP so that resolving the proxy's own name
// cannot influence the assertion.
func TestDialSMTP_ProxyBypassesCustomResolver(t *testing.T) {
domain := "github.com:25"
proxyURI := "socks5://127.0.0.1:1080"
timeout := 5 * time.Second
var called atomic.Bool
customResolver := &net.Resolver{
PreferGo: true,
Dial: func(ctx context.Context, network, address string) (net.Conn, error) {
called.Store(true)
return nil, errors.New("custom resolver dial invoked")
},
}
ret, err := dialSMTP(domain, proxyURI, customResolver, timeout, timeout)
assert.Nil(t, ret)
require.Error(t, err)
assert.False(t, called.Load())
}
func TestPreferredDialError(t *testing.T) {
network := errors.New("dial tcp 1.2.3.4:25: connect: connection refused")
dns := &net.DNSError{Err: "no such host", Name: "mx.example.invalid", IsNotFound: true}
reply := &textproto.Error{Code: 550, Msg: "5.7.1 Service unavailable, Client host blocked using Spamhaus."}
rendered := errors.New("421 4.7.0 Too many concurrent connections")
t.Run("prefers a server reply over a network failure", func(tt *testing.T) {
// the network failure arrives first because it fails fastest
assert.Same(tt, reply, preferredDialError([]error{network, reply}))
})
t.Run("prefers a server reply over a DNS failure", func(tt *testing.T) {
assert.Same(tt, reply, preferredDialError([]error{dns, reply}))
})
t.Run("recognises a reply that was already rendered to a string", func(tt *testing.T) {
assert.Same(tt, rendered, preferredDialError([]error{network, rendered}))
})
t.Run("falls back to the first error when no server replied", func(tt *testing.T) {
assert.Same(tt, network, preferredDialError([]error{network, dns}))
})
t.Run("single error", func(tt *testing.T) {
assert.Same(tt, network, preferredDialError([]error{network}))
})
}
func TestHasSMTPReply(t *testing.T) {
cases := []struct {
name string
err error
want bool
}{
{"textproto error", &textproto.Error{Code: 550, Msg: "no such user"}, true},
{"rendered reply", errors.New("452 4.5.3 Recipients belong to multiple regions"), true},
{"connection refused", errors.New("dial tcp 1.2.3.4:25: connect: connection refused"), false},
{"dns miss", &net.DNSError{Err: "no such host"}, false},
{"too short to carry a code", errors.New("no"), false},
{"leading digits that are not a reply code", errors.New("103.26.221.42 unreachable"), false},
{"no MX records", errors.New("No MX records found"), false},
}
for _, c := range cases {
test := c
t.Run(test.name, func(tt *testing.T) {
assert.Equal(tt, test.want, hasSMTPReply(test.err))
})
}
}
// The API-verifier branch returns before our own catch-all probe runs, so the
// outcome has to come from what the verifier itself reported. Leaving it at
// catchAllNotRun would turn a verifier's CatchAll: true into reachable "no".
func TestCatchAllFromResult(t *testing.T) {
assert.Equal(t, catchAllAccepted, catchAllFromResult(&SMTP{CatchAll: true}))
assert.Equal(t, catchAllRefused, catchAllFromResult(&SMTP{CatchAll: false}))
assert.Equal(t, catchAllRefused, catchAllFromResult(nil))
}
verifier.go
+300
View File
@@ -0,0 +1,300 @@
package emailverifier
import (
"fmt"
"net"
"time"
)
// Verifier is an email verifier. Create one by calling NewVerifier
type Verifier struct {
smtpCheckEnabled bool // SMTP check enabled or disabled (disabled by default)
catchAllCheckEnabled bool // SMTP catchAll check enabled or disabled (enabled by default)
domainSuggestEnabled bool // whether suggest a most similar correct domain or not (disabled by default)
gravatarCheckEnabled bool // gravatar check enabled or disabled (disabled by default)
fromEmail string // name to use in the `EHLO:` SMTP command, defaults to "user@example.org"
helloName string // email to use in the `MAIL FROM:` SMTP command. defaults to `localhost`
schedule *schedule // schedule represents a job schedule
proxyURI string // use a SOCKS5 proxy to verify the email,
apiVerifiers map[string]smtpAPIVerifier // per-vendor API verifiers; no built-in vendors currently, contributions are welcomed.
resolver *net.Resolver // resolver used to perform MX and SMTP DNS lookups; nil means net.DefaultResolver, read at call time
// Timeouts
connectTimeout time.Duration // Timeout for establishing connections
operationTimeout time.Duration // Timeout for SMTP operations (e.g., EHLO, MAIL FROM, etc.)
}
// Result is the result of Email Verification
type Result struct {
Email string `json:"email"` // passed email address
Reachable string `json:"reachable"` // an enumeration to describe whether the recipient address is real
Syntax Syntax `json:"syntax"` // details about the email address syntax
SMTP *SMTP `json:"smtp"` // details about the SMTP response of the email
Gravatar *Gravatar `json:"gravatar"` // whether or not have gravatar for the email
Suggestion string `json:"suggestion"` // domain suggestion when domain is misspelled
Disposable bool `json:"disposable"` // is this a DEA (disposable email address)
RoleAccount bool `json:"role_account"` // is account a role-based account
Free bool `json:"free"` // is domain a free email domain
HasMxRecords bool `json:"has_mx_records"` // whether or not MX-Records for the domain
}
// init loads disposable_domain meta data to disposableSyncDomains which are safe for concurrent use
func init() {
for d := range disposableDomains {
disposableSyncDomains.Store(d, struct{}{})
}
}
// NewVerifier creates a new email verifier
func NewVerifier() *Verifier {
return &Verifier{
fromEmail: defaultFromEmail,
helloName: defaultHelloName,
catchAllCheckEnabled: true,
apiVerifiers: map[string]smtpAPIVerifier{},
connectTimeout: 10 * time.Second,
operationTimeout: 10 * time.Second,
}
}
// Verify performs address, misc, mx and smtp checks
func (v *Verifier) Verify(email string) (*Result, error) {
ret := Result{
Email: email,
Reachable: reachableUnknown,
}
syntax := v.ParseAddress(email)
ret.Syntax = syntax
if !syntax.Valid {
return &ret, nil
}
ret.Free = v.IsFreeDomain(syntax.Domain)
ret.RoleAccount = v.IsRoleAccount(syntax.Username)
ret.Disposable = v.IsDisposable(syntax.Domain)
if v.domainSuggestEnabled {
ret.Suggestion = v.SuggestDomain(syntax.Domain)
}
// If the domain name is disposable, mx and smtp are not checked.
if ret.Disposable {
return &ret, nil
}
mx, err := v.CheckMX(syntax.Domain)
if err != nil {
errStr := err.Error()
if insContains(errStr, "no such host") {
ret.Reachable = reachableNo
return &ret, newLookupError(ErrNoSuchHost, errStr).withCause(err)
}
return &ret, err
}
ret.HasMxRecords = mx.HasMXRecord
smtp, catchAll, err := v.checkSMTP(syntax.Domain, syntax.Username)
// Kept whether or not the check succeeded. A failed exchange still says how
// far it got, which is the difference between a host that never answered and
// one that answered and refused our sender.
ret.SMTP = smtp
if err != nil {
return &ret, err
}
ret.Reachable = v.calculateReachable(smtp, catchAll)
if v.gravatarCheckEnabled {
gravatar, err := v.CheckGravatar(email)
if err != nil {
return &ret, err
}
ret.Gravatar = gravatar
}
return &ret, nil
}
// AddDisposableDomains adds additional domains as disposable domains.
func (v *Verifier) AddDisposableDomains(domains []string) *Verifier {
for _, d := range domains {
additionalDisposableDomains.Store(d, struct{}{})
disposableSyncDomains.Store(d, struct{}{})
}
return v
}
// EnableGravatarCheck enables check gravatar,
// we don't check gravatar by default
func (v *Verifier) EnableGravatarCheck() *Verifier {
v.gravatarCheckEnabled = true
return v
}
// DisableGravatarCheck disables check gravatar,
func (v *Verifier) DisableGravatarCheck() *Verifier {
v.gravatarCheckEnabled = false
return v
}
// EnableSMTPCheck enables check email by smtp,
// for most ISPs block outgoing SMTP requests through port 25, to prevent spam,
// we don't check smtp by default
func (v *Verifier) EnableSMTPCheck() *Verifier {
v.smtpCheckEnabled = true
return v
}
// EnableAPIVerifier activates an API-based existence check for the given vendor.
// ** Please know ** that this is a tricky way (but relatively stable) to check if target vendor's email exists.
// If you use this feature in a production environment, please ensure that you have sufficient backup measures in place, as this may encounter rate limiting or other API issues.
//
// There are currently no built-in vendors: the Yahoo verifier was removed once its
// endpoint stopped existing (see https://github.com/AfterShip/email-verifier/issues/195),
// following the same fate as the Gmail verifier removed in #113. This remains an
// extension point; contributions adding a working vendor are welcome.
func (v *Verifier) EnableAPIVerifier(name string) error {
return fmt.Errorf("unsupported to enable the API verifier for vendor: %s", name)
}
func (v *Verifier) DisableAPIVerifier(name string) {
delete(v.apiVerifiers, name)
}
// DisableSMTPCheck disables check email by smtp
func (v *Verifier) DisableSMTPCheck() *Verifier {
v.smtpCheckEnabled = false
return v
}
// EnableCatchAllCheck enables the catchAll check by smtp. It is on by default;
// this undoes a previous DisableCatchAllCheck.
func (v *Verifier) EnableCatchAllCheck() *Verifier {
v.catchAllCheckEnabled = true
return v
}
// DisableCatchAllCheck disables catchAll check by smtp
func (v *Verifier) DisableCatchAllCheck() *Verifier {
v.catchAllCheckEnabled = false
return v
}
// EnableDomainSuggest will suggest a most similar correct domain when domain misspelled
func (v *Verifier) EnableDomainSuggest() *Verifier {
v.domainSuggestEnabled = true
return v
}
// DisableDomainSuggest will not suggest anything
func (v *Verifier) DisableDomainSuggest() *Verifier {
v.domainSuggestEnabled = false
return v
}
// EnableAutoUpdateDisposable enables update disposable domains automatically
func (v *Verifier) EnableAutoUpdateDisposable() *Verifier {
v.stopCurrentSchedule()
// fetch latest disposable domains before next schedule
_ = updateDisposableDomains(disposableDataURL)
// update disposable domains records daily
v.schedule = newSchedule(24*time.Hour, updateDisposableDomains, disposableDataURL)
v.schedule.start()
return v
}
// DisableAutoUpdateDisposable stops previously started schedule job
func (v *Verifier) DisableAutoUpdateDisposable() *Verifier {
v.stopCurrentSchedule()
return v
}
// FromEmail sets the emails to use in the `MAIL FROM:` smtp command
func (v *Verifier) FromEmail(email string) *Verifier {
v.fromEmail = email
return v
}
// HelloName sets the name to use in the `EHLO:` SMTP command
func (v *Verifier) HelloName(domain string) *Verifier {
v.helloName = domain
return v
}
// Proxy sets a SOCKS5 proxy to verify the email,
// proxyURI should be in the format: "socks5://user:password@127.0.0.1:1080".
// The protocol could be socks5, socks4 and socks4a.
//
// Only the scheme, credentials and host are read; a query string is ignored, so
// a "?timeout=5s" has no effect. Use ConnectTimeout and OperationTimeout.
//
// The proxy carries the connection to the mail server, not DNS: MX lookups
// still go out from the local machine. See Resolver for routing those through
// the proxy too.
func (v *Verifier) Proxy(proxyURI string) *Verifier {
v.proxyURI = proxyURI
return v
}
// Resolver sets a custom DNS resolver to use for MX and SMTP host lookups,
// allowing users to configure custom DNS servers instead of relying on the
// system's default resolver (e.g. /etc/resolv.conf). Passing nil restores
// the default resolver (net.DefaultResolver).
func (v *Verifier) Resolver(resolver *net.Resolver) *Verifier {
v.resolver = resolver
return v
}
// dnsResolver returns the resolver to use for lookups. net.DefaultResolver is
// read here rather than captured in NewVerifier so that callers who replace the
// global keep the behaviour they had before this option existed -- the previous
// code called net.LookupMX, which reads net.DefaultResolver on every call.
func (v *Verifier) dnsResolver() *net.Resolver {
if v.resolver == nil {
return net.DefaultResolver
}
return v.resolver
}
// ConnectTimeout sets the timeout for establishing connections.
func (v *Verifier) ConnectTimeout(timeout time.Duration) *Verifier {
v.connectTimeout = timeout
return v
}
// OperationTimeout sets the timeout for SMTP operations (e.g., EHLO, MAIL FROM, etc.).
func (v *Verifier) OperationTimeout(timeout time.Duration) *Verifier {
v.operationTimeout = timeout
return v
}
// calculateReachable turns the SMTP observations into a verdict. Callers reach
// it only from Verify, which always has a username: a syntactically valid
// address cannot have an empty local part, so the address itself was always
// probed unless the catch-all probe returned first.
func (v *Verifier) calculateReachable(s *SMTP, catchAll catchAllOutcome) string {
if !v.smtpCheckEnabled {
return reachableUnknown
}
if s.Deliverable {
return reachableYes
}
// Deliberately not s.CatchAll, which is also true when the probe never ran
// or could not conclude. Only a probe that established a catch-all domain,
// or one that returned before the address was probed, leaves the address
// unverifiable; otherwise the address was refused and that is a definite no.
if catchAll == catchAllAccepted || catchAll == catchAllInconclusive {
return reachableUnknown
}
return reachableNo
}
// stopCurrentSchedule stops current running schedule (if exists)
func (v *Verifier) stopCurrentSchedule() {
if v.schedule != nil {
v.schedule.stop()
}
}