Testability Strategy #28

@xinity commented on GitHub (Nov 4, 2016):

👍 for go-gitea/proposals

@xinity commented on GitHub (Nov 4, 2016): :+1: for go-gitea/proposals

GiteaMirror commented

@tboerger commented on GitHub (Nov 4, 2016):

And what do you want to propose? :D

@tboerger commented on GitHub (Nov 4, 2016): And what do you want to propose? :D

GiteaMirror commented

@andreynering commented on GitHub (Nov 5, 2016):

I think this is a big topic, so we should break it in small parts.

Unit tests

First of all we should have more unit tests. These tests should be memory only (e.g.: they shouldn't touch the database, git, and if possible even the filesystem). They are tests for isolated parts of the codebase.

Integration testing

This is harder, here we have to simulate HTTP requests and check if the behavior is correct. I think there are two main difficulties:

How to deal with a database

Mocking the database
We can test against a real database

I prefer to use a real database, since mocking will not catch problems like wrong SQL sintax, constraints violations, etc. #86 is a example that uses this approach. We can use a memory database for performance.

How to deal with Git

We can mock the interface that talk to git
We can write scripts to generate repositories and use real Git

In this case I prefer mocking since it will be easier than the alternative.

After we have a consensus it should be write in a formal document.

@andreynering commented on GitHub (Nov 5, 2016): I think this is a big topic, so we should break it in small parts. ### Unit tests First of all we should have more unit tests. These tests should be memory only (e.g.: they shouldn't touch the database, git, and if possible even the filesystem). They are tests for isolated parts of the codebase. ### Integration testing This is harder, here we have to simulate HTTP requests and check if the behavior is correct. I think there are two main difficulties: #### How to deal with a database - Mocking the database - We can test against a real database I prefer to use a real database, since mocking will not catch problems like wrong SQL sintax, constraints violations, etc. #86 is a example that uses this approach. We can use a memory database for performance. #### How to deal with Git - We can mock the interface that talk to git - We can write scripts to generate repositories and use real Git In this case I prefer mocking since it will be easier than the alternative. --- After we have a consensus it should be write in a formal document.

GiteaMirror commented

@strk commented on GitHub (Nov 5, 2016):

+1 for real database, and ideally against all the supported and available databases systems
And also I think using real git would be better

Should this go under go-gitea/proposals, btw ?

@strk commented on GitHub (Nov 5, 2016): +1 for real database, and ideally against all the supported and available databases systems And also I think using real git would be better Should this go under go-gitea/proposals, btw ?

GiteaMirror commented

@metalmatze commented on GitHub (Nov 5, 2016):

Just want to add that I like to be mostly stdlib for unit tests.
Nevertheless I find https://godoc.org/github.com/stretchr/testify/assert extremely useful to prevent a lot of boilerplate.

val, err := somefunc()
if err != nil {
    t.Error(err)
}
if val != "foobar" {
    t.Error("val is not foobar")
}

would become:

val, err := somefunc()
assert.Nil(t, err)
assert.Equals(t, "foobar", val)

That's it.

@metalmatze commented on GitHub (Nov 5, 2016): Just want to add that I like to be mostly stdlib for unit tests. Nevertheless I find https://godoc.org/github.com/stretchr/testify/assert extremely useful to prevent a lot of boilerplate. ``` go val, err := somefunc() if err != nil { t.Error(err) } if val != "foobar" { t.Error("val is not foobar") } ``` would become: ``` go val, err := somefunc() assert.Nil(t, err) assert.Equals(t, "foobar", val) ``` That's it.

GiteaMirror commented

@andreynering commented on GitHub (Nov 5, 2016):

@metalmatze I agree. Go Convey was the convention on Gogs, but I also prefer testify/assert. That's another thing we should decide.

@andreynering commented on GitHub (Nov 5, 2016): @metalmatze I agree. Go Convey was the convention on Gogs, but I also prefer testify/assert. That's another thing we should decide.

GiteaMirror commented

@makhov commented on GitHub (Nov 5, 2016):

Convey is awful.
@metalmatze, note that testify/assert has methods assert.Error and assert.NoError to test errors

@makhov commented on GitHub (Nov 5, 2016): Convey is awful. @metalmatze, note that `testify/assert` has methods `assert.Error` and `assert.NoError` to test errors

GiteaMirror commented

@metalmatze commented on GitHub (Nov 5, 2016):

@makhov Even better! 😄 👍

@metalmatze commented on GitHub (Nov 5, 2016): @makhov Even better! :smile: :+1:

GiteaMirror commented

@lunny commented on GitHub (Nov 6, 2016):

Unit Tests first, that's great idea. And let's send more unit tests PRs.
About the unit test framework, we have to determine one ASAP.

@makhov, do you have time on code review PRs? If you like, I want to invite you to maintainers team.

@lunny commented on GitHub (Nov 6, 2016): Unit Tests first, that's great idea. And let's send more unit tests PRs. About the unit test framework, we have to determine one ASAP. @makhov, do you have time on code review PRs? If you like, I want to invite you to maintainers team.

GiteaMirror commented

@makhov commented on GitHub (Nov 6, 2016):

@lunny right after somebody explain me what happens :)

@makhov commented on GitHub (Nov 6, 2016): @lunny right after somebody explain me what happens :)

GiteaMirror commented

@lunny commented on GitHub (Nov 6, 2016):

@makhov Welcome back! I have moved you from Advisor Team to Maintainers Team.

@lunny commented on GitHub (Nov 6, 2016): @makhov Welcome back! I have moved you from Advisor Team to Maintainers Team.

GiteaMirror commented

@odinuge commented on GitHub (Nov 6, 2016):

Isn't pure golang testing an equally good approach to unit testing? The assertion approach works, but in my opinion, the normal go-one works better. Eg. it forces the user testing to handle errors, instead of just panicking.

Ofc. they both comes with some different pros and cons, but the most important thing (as @lynny states) is to agree on "one" way to do it, and to actually write the tests! :)

@odinuge commented on GitHub (Nov 6, 2016): Isn't pure golang [testing](https://golang.org/pkg/testing/) an equally good approach to unit testing? The `assertion` approach works, but in my opinion, the normal go-one works better. Eg. it forces the user testing to handle errors, instead of just panicking. Ofc. they both comes with some different pros and cons, but the most important thing (as @lynny states) is to agree on "one" way to do it, and to _actually_ write the tests! :)

GiteaMirror commented

@makhov commented on GitHub (Nov 6, 2016):

We use testing with testify/assert and it works for us.

@makhov commented on GitHub (Nov 6, 2016): We use `testing` with `testify/assert` and it works for us.

GiteaMirror commented

@strk commented on GitHub (Nov 6, 2016):

testify/assert works for me (never used), but have no objection if anyone wants to provide bare testing tests too

@strk commented on GitHub (Nov 6, 2016): testify/assert works for me (never used), but have no objection if anyone wants to provide bare `testing` tests too

GiteaMirror commented

@lunny commented on GitHub (Nov 7, 2016):

So let's use github.com/stretchr/testify, and anything else needed for unit tests?

@lunny commented on GitHub (Nov 7, 2016): So let's use github.com/stretchr/testify, and anything else needed for unit tests?

GiteaMirror commented

@tboerger commented on GitHub (Nov 7, 2016):

What about goblin? I really like this lib, especially with my BDD background on ruby from the past ;)

@tboerger commented on GitHub (Nov 7, 2016): What about goblin? I really like this lib, especially with my BDD background on ruby from the past ;)

GiteaMirror commented

@metalmatze commented on GitHub (Nov 7, 2016):

I think something like Goblin could be pretty good for integration tests. But for simple Unit Tests I'd prefer something very basic like stdlib with testify.

@metalmatze commented on GitHub (Nov 7, 2016): I think something like Goblin could be pretty good for integration tests. But for simple Unit Tests I'd prefer something very basic like stdlib with testify.

GiteaMirror commented

@bkcsoft commented on GitHub (Nov 7, 2016):

@andreynering

First of all we should have more unit tests. These tests should be memory only (e.g.: they shouldn't touch the database, git, and if possible even the filesystem).

I disagree, unit-tests are optimal for testing db/git/fs, since they test a specific thing and not End-To-End (Integration tests) they would run fairly fast, and only once per code-path (which is almost impossible with e2e-tests) 🙂

@bkcsoft commented on GitHub (Nov 7, 2016): @andreynering > First of all we should have more unit tests. These tests should be memory only (e.g.: they shouldn't touch the database, git, and if possible even the filesystem). I disagree, unit-tests are optimal for testing db/git/fs, since they test a specific thing and not End-To-End (Integration tests) they would run fairly fast, and _only once per code-path_ (which is almost impossible with e2e-tests) :slightly_smiling_face:

GiteaMirror commented

@andreynering commented on GitHub (Nov 8, 2016):

@bkcsoft OK, I agree, I think it's OK to touch Git or the DB with you are actually testing things that touch them.

@andreynering commented on GitHub (Nov 8, 2016): @bkcsoft OK, I agree, I think it's OK to touch Git or the DB with you are actually testing things that touch them.

GiteaMirror commented

@lunny commented on GitHub (Feb 11, 2017):

I remember one PR has moved goconvey already?

@lunny commented on GitHub (Feb 11, 2017): I remember one PR has moved goconvey already?

GiteaMirror commented

@bkcsoft commented on GitHub (Feb 12, 2017):

@lunny As previously mentioned, unit-tests will still use testify/assert while integration-tests could use goconvey (even though I don't like that lib...)

@bkcsoft commented on GitHub (Feb 12, 2017): @lunny As previously mentioned, unit-tests will still use `testify/assert` while integration-tests could use `goconvey` (even though I don't like that lib...)

GiteaMirror commented

@andreynering commented on GitHub (Feb 12, 2017):

I vote to use only testify/assert.

@andreynering commented on GitHub (Feb 12, 2017): I vote to use only testify/assert.

GiteaMirror commented

@lunny commented on GitHub (Feb 12, 2017):

^

@lunny commented on GitHub (Feb 12, 2017): ^

GiteaMirror commented