# How to log bad data in an import script

**URL:** https://meta.discourse.org/t/how-to-log-bad-data-in-an-import-script/49034
**Category:** Development
**Created:** [22 augustus 2016 om 23:41 UTC](https://meta.discourse.org/t/how-to-log-bad-data-in-an-import-script/49034 "2016-08-22T23:41:38Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![pfaffman](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pfaffman/32/120154_2.png) [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [22 augustus 2016 om 23:41 UTC](https://meta.discourse.org/t/how-to-log-bad-data-in-an-import-script/49034/1 "2016-08-22T23:41:38Z")

</div>

I’ve made considerable improvements to `mbox.rb`, the mbox email importer. I’m importing 230K messages, many of which have had the email addresses broken in one way or another. The script used to just crap out if something went wrong; I’ve added code to let the user know what file is being processed and print error messages including the bad address and the message-id when email addresses are bad and the message ID and who message when something else goes wrong.

`print_status` does a lovely job of, well, printing the status, but it’s less pretty when I `put` error messages. Also, when there are hundreds of lines of lines of such messages, having them in a terminal can be a bit unwieldy (especially if you switch sessions in `tmux` and lose the scroll-back buffer). Maybe what I should do is send stuff to stderr? Or should I just not worry?

---

<div class="post-metadata">

### Author: ![mpalmer](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/mpalmer/32/45740_2.png) [@mpalmer](https://meta.discourse.org/u/mpalmer)
#### Post date: [23 augustus 2016 om 00:15 UTC](https://meta.discourse.org/t/how-to-log-bad-data-in-an-import-script/49034/2 "2016-08-23T00:15:32Z")

</div>

Yes, problems and errors should be sent to stderr, rather than stdout. The user can then capture that output to a file for later analysis, with `2>/tmp/import_errors.log`, without interrupting the progress and status information.

---

<div class="post-metadata">

### Author: ![pfaffman](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pfaffman/32/120154_2.png) [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [23 augustus 2016 om 00:58 UTC](https://meta.discourse.org/t/how-to-log-bad-data-in-an-import-script/49034/3 "2016-08-23T00:58:23Z")

</div>

Is there a way we like? Otherwise, I’ll just Google “ruby output stderr” and see what comes up. 🙂

---

<div class="post-metadata">

### Author: ![mpalmer](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/mpalmer/32/45740_2.png) [@mpalmer](https://meta.discourse.org/u/mpalmer)
#### Post date: [23 augustus 2016 om 01:14 UTC](https://meta.discourse.org/t/how-to-log-bad-data-in-an-import-script/49034/4 "2016-08-23T01:14:01Z")

</div>

```
$stderr.puts "I'm in a stderr!"

```

Probably worth wrapping it up in some sort of `print_error` method, for clarity and overridability.
