Disable fragile tests for now by MorrisJobke · Pull Request #14541 · nextcloud/server

MorrisJobke · 2019-03-05T13:26:20Z

Those are the ones that fail on all the branches and just hide the real errors. After merging this I will open a revert of this PR to properly analyze where this broke.

Signed-off-by: Morris Jobke <hey@morrisjobke.de>

faily-bot · 2019-03-05T14:36:46Z

🤖 beep boop beep 🤖

Here are the logs for the failed build:

Status of 16768: failure

TESTS=acceptance, TESTS-ACCEPTANCE=app-comments
- tests/acceptance/features/app-comments.feature:10

Show full log

  Scenario: open the comments for a different file           # /drone/src/github.com/nextcloud/server/tests/acceptance/features/app-comments.feature:10
    Given I am logged in                                     # LoginPageContext::iAmLoggedIn()
    And I create a new folder named "Folder"                 # FileListContext::iCreateANewFolderNamed()
    And I open the details view for "welcome.txt"            # FileListContext::iOpenTheDetailsViewFor()
    And I open the "Comments" tab in the details view        # FilesAppContext::iOpenTheTabInTheDetailsView()
    And I create a new comment with "Hello world" as message # CommentsAppContext::iCreateANewCommentWithAsMessage()
    And I see a comment with "Hello world" as message        # CommentsAppContext::iSeeACommentWithAsMessage()
      │ Comment with text "Hello world" in details view in Files app visibility could not be got
      │ Exception message: Element not found with xpath, (((//html/.//*[./@id = 'app-sidebar'])[1]/descendant-or-self::ul[@class and contains(concat(' ', normalize-space(@class), ' '), ' comments ')])[1]//div[normalize-space() = 'Hello world']/ancestor::li)[1]
      │ 
      │ Unable to locate element: {"method":"xpath","selector":"(((//html/.//*[./@id = 'app-sidebar'])[1]/descendant-or-self::ul[@class and contains(concat(' ', normalize-space(@class), ' '), ' comments ')])[1]//div[normalize-space() = 'Hello world']/ancestor::li)[1]"}
      │ For documentation on this error, please visit: http://seleniumhq.org/exceptions/no_such_element.html
      │ Build info: version: '2.53.1', revision: 'a36b8b1', time: '2016-06-30 17:37:03'
      │ System info: host: '8bef4bbcdef8', ip: '172.17.0.3', os.name: 'Linux', os.arch: 'amd64', os.version: '4.4.0-141-generic', java.version: '1.8.0_91'
      │ Driver info: driver.version: unknown
      │ Trying again
      │ 
      Comment with text "Hello world" in details view in Files app could not be found after 100 seconds (NoSuchElementException)
    When I open the details view for "Folder"                # FileListContext::iOpenTheDetailsViewFor()

TESTS=acceptance, TESTS-ACCEPTANCE=app-files
- tests/acceptance/features/app-files.feature:60
- tests/acceptance/features/app-files.feature:133

Show full log

  Scenario: viewing a favorite file in its folder does not prevent opening the details view in "All files" section # /drone/src/github.com/nextcloud/server/tests/acceptance/features/app-files.feature:60
    Given I am logged in                                                                                           # LoginPageContext::iAmLoggedIn()
    And I mark "welcome.txt" as favorite                                                                           # FileListContext::iMarkAsFavorite()
    And I see that "welcome.txt" is marked as favorite                                                             # FileListContext::iSeeThatIsMarkedAsFavorite()
    And I open the "Favorites" section                                                                             # AppNavigationContext::iOpenTheSection()
    And I open the details view for "welcome.txt"                                                                  # FileListContext::iOpenTheDetailsViewFor()
      Row for file welcome.txt in file list could not be found after 100 seconds
      File actions menu button for file welcome.txt in file list could not be found after 100 seconds (NoSuchElementException)
    And I see that the details view is open                                                                        # FilesAppContext::iSeeThatTheDetailsViewIsOpen()
    And I view "welcome.txt" in folder                                                                             # FileListContext::iViewInFolder()
    And I see that the current section is "All files"                                                              # AppNavigationContext::iSeeThatTheCurrentSectionIs()
    When I open the details view for "welcome.txt"                                                                 # FileListContext::iOpenTheDetailsViewFor()
    Then I see that the details view is open                                                                       # FilesAppContext::iSeeThatTheDetailsViewIsOpen()

  Scenario: show deleted files for a second time                      # /drone/src/github.com/nextcloud/server/tests/acceptance/features/app-files.feature:133
    Given I am logged in                                              # LoginPageContext::iAmLoggedIn()
    And I open the "Deleted files" section                            # AppNavigationContext::iOpenTheSection()
    And I see that the current section is "Deleted files"             # AppNavigationContext::iSeeThatTheCurrentSectionIs()
    And I open the "All files" section                                # AppNavigationContext::iOpenTheSection()
    And I see that the current section is "All files"                 # AppNavigationContext::iSeeThatTheCurrentSectionIs()
    And I delete "welcome.txt"                                        # FileListContext::iDelete()
    When I open the "Deleted files" section                           # AppNavigationContext::iOpenTheSection()
    Then I see that the current section is "Deleted files"            # AppNavigationContext::iSeeThatTheCurrentSectionIs()
    Then I see that the file list contains a file named "welcome.txt" # FileListContext::iSeeThatTheFileListContainsAFileNamed()
      Row for file welcome.txt in file list could not be found after 100 seconds (NoSuchElementException)

TESTS=acceptance, TESTS-ACCEPTANCE=app-files-tags
- tests/acceptance/features/app-files-tags.feature:70

Show full log

  Scenario: remove tags using the dropdown in the details view                                  # /drone/src/github.com/nextcloud/server/tests/acceptance/features/app-files-tags.feature:70
    Given I am logged in as the admin                                                           # LoginPageContext::iAmLoggedInAsTheAdmin()
    And I visit the settings page                                                               # SettingsMenuContext::iVisitTheSettingsPage()
    And I open the "Tag management" section                                                     # AppNavigationContext::iOpenTheSection()
    And I see that the button to select tags is shown                                           # SettingsContext::iSeeThatTheButtonToSelectTagsIsShown()
    And I create the tag "tag1" in the settings                                                 # SettingsContext::iCreateTheTagInTheSettings()
    And I create the tag "tag2" in the settings                                                 # SettingsContext::iCreateTheTagInTheSettings()
    And I create the tag "tag3" in the settings                                                 # SettingsContext::iCreateTheTagInTheSettings()
    And I create the tag "tag4" in the settings                                                 # SettingsContext::iCreateTheTagInTheSettings()
    And I see that the dropdown for tags in the settings eventually contains the tag "tag1"     # SettingsContext::iSeeThatTheDropdownForTagsInTheSettingsEventuallyContainsTheTag()
    And I see that the dropdown for tags in the settings eventually contains the tag "tag2"     # SettingsContext::iSeeThatTheDropdownForTagsInTheSettingsEventuallyContainsTheTag()
    And I see that the dropdown for tags in the settings eventually contains the tag "tag3"     # SettingsContext::iSeeThatTheDropdownForTagsInTheSettingsEventuallyContainsTheTag()
    And I see that the dropdown for tags in the settings eventually contains the tag "tag4"     # SettingsContext::iSeeThatTheDropdownForTagsInTheSettingsEventuallyContainsTheTag()
    And I log out                                                                               # SettingsMenuContext::iLogOut()
    And I am logged in                                                                          # LoginPageContext::iAmLoggedIn()
    And I open the details view for "welcome.txt"                                               # FileListContext::iOpenTheDetailsViewFor()
    And I open the input field for tags in the details view                                     # FilesAppContext::iOpenTheInputFieldForTagsInTheDetailsView()
    And I check the tag "tag2" in the dropdown for tags in the details view                     # FilesAppContext::iCheckTheTagInTheDropdownForTagsInTheDetailsView()
    And I check the tag "tag4" in the dropdown for tags in the details view                     # FilesAppContext::iCheckTheTagInTheDropdownForTagsInTheDetailsView()
    And I check the tag "tag3" in the dropdown for tags in the details view                     # FilesAppContext::iCheckTheTagInTheDropdownForTagsInTheDetailsView()
    When I uncheck the tag "tag2" in the dropdown for tags in the details view                  # FilesAppContext::iUncheckTheTagInTheDropdownForTagsInTheDetailsView()
    And I uncheck the tag "tag4" in the dropdown for tags in the details view                   # FilesAppContext::iUncheckTheTagInTheDropdownForTagsInTheDetailsView()
    Then I see that the tag "tag2" in the dropdown for tags in the details view is not checked  # FilesAppContext::iSeeThatTheTagInTheDropdownForTagsInTheDetailsViewIsNotChecked()
    And I see that the tag "tag4" in the dropdown for tags in the details view is not checked   # FilesAppContext::iSeeThatTheTagInTheDropdownForTagsInTheDetailsViewIsNotChecked()
    And I see that the tag "tag3" in the dropdown for tags in the details view is checked       # FilesAppContext::iSeeThatTheTagInTheDropdownForTagsInTheDetailsViewIsChecked()
    And I see that the input field for tags in the details view does not contain the tag "tag2" # FilesAppContext::iSeeThatTheInputFieldForTagsInTheDetailsViewDoesNotContainTheTag()
      Failed asserting that false is true.
    And I see that the input field for tags in the details view does not contain the tag "tag4" # FilesAppContext::iSeeThatTheInputFieldForTagsInTheDetailsViewDoesNotContainTheTag()
    And I see that the input field for tags in the details view contains the tag "tag3"         # FilesAppContext::iSeeThatTheInputFieldForTagsInTheDetailsViewContainsTheTag()

TESTS=acceptance, TESTS-ACCEPTANCE=apps
- tests/acceptance/features/apps.feature:75

Show full log

  Scenario: View app details for app store apps   # /drone/src/github.com/nextcloud/server/tests/acceptance/features/apps.feature:75
    Given I act as Jane                           # ActorContext::iActAs()
    And I am logged in as the admin               # LoginPageContext::iAmLoggedInAsTheAdmin()
    And I open the Apps management                # SettingsMenuContext::iOpenTheAppsManagement()
    And I open the "Tools" section                # AppNavigationContext::iOpenTheSection()
      Tools section item in App Navigation could not be found after 100 seconds (NoSuchElementException)
    When I click on the "Antivirus for files" app # AppsManagementContext::iClickOnTheApp()
    Then I see that the app details are shown     # AppsManagementContext::iSeeThatTheAppDetailsAreShown()

ChristophWurst

Fine by me. But will CI pass on this one? 🙈

MorrisJobke · 2019-03-05T16:18:18Z

Fine by me. But will CI pass on this one? 🙈

Now there are other failing ones 😭 Let me retrigger the job.

MorrisJobke · 2019-03-05T16:18:40Z

https://drone.nextcloud.com/nextcloud/server/16789

faily-bot · 2019-03-05T18:47:46Z

🤖 beep boop beep 🤖

Here are the logs for the failed build:

Status of 16789: failure

TESTS=acceptance, TESTS-ACCEPTANCE=app-comments

tests/acceptance/features/app-comments.feature:10
tests/acceptance/features/app-comments.feature:222

Show full log

  Scenario: open the comments for a different file           # /drone/src/github.com/nextcloud/server/tests/acceptance/features/app-comments.feature:10
    Given I am logged in                                     # LoginPageContext::iAmLoggedIn()
    And I create a new folder named "Folder"                 # FileListContext::iCreateANewFolderNamed()
    And I open the details view for "welcome.txt"            # FileListContext::iOpenTheDetailsViewFor()
    And I open the "Comments" tab in the details view        # FilesAppContext::iOpenTheTabInTheDetailsView()
    And I create a new comment with "Hello world" as message # CommentsAppContext::iCreateANewCommentWithAsMessage()
    And I see a comment with "Hello world" as message        # CommentsAppContext::iSeeACommentWithAsMessage()
      Comment with text "Hello world" in details view in Files app could not be found after 100 seconds (NoSuchElementException)
    When I open the details view for "Folder"                # FileListContext::iOpenTheDetailsViewFor()

  Scenario: search a comment                                      # /drone/src/github.com/nextcloud/server/tests/acceptance/features/app-comments.feature:222
    Given I am logged in                                          # LoginPageContext::iAmLoggedIn()
    And I open the details view for "welcome.txt"                 # FileListContext::iOpenTheDetailsViewFor()
    And I open the "Comments" tab in the details view             # FilesAppContext::iOpenTheTabInTheDetailsView()
    And I create a new comment with "Hello world" as message      # CommentsAppContext::iCreateANewCommentWithAsMessage()
    And I see a comment with "Hello world" as message             # CommentsAppContext::iSeeACommentWithAsMessage()
    When I search for "hello"                                     # SearchContext::iSearchFor()
    Then I see that the search result 1 is "user0Hello world"     # SearchContext::iSeeThatTheSearchResultIs()
      Failed asserting that two strings are equal.
      --- Expected
      +++ Actual
      @@ @@
      -'user0Hello world'
      +'Hello world'
    And I see that the search result 1 was found in "welcome.txt" # SearchContext::iSeeThatTheSearchResultWasFoundIn()

TESTS=acceptance, TESTS-ACCEPTANCE=app-files

tests/acceptance/features/app-files.feature:72
tests/acceptance/features/app-files.feature:90
tests/acceptance/features/app-files.feature:126

Show full log

  Scenario: show recent files                                                 # /drone/src/github.com/nextcloud/server/tests/acceptance/features/app-files.feature:72
    Given I am logged in                                                      # LoginPageContext::iAmLoggedIn()
    And I create a new folder named "Folder just created"                     # FileListContext::iCreateANewFolderNamed()
    When I open the "Recent" section                                          # AppNavigationContext::iOpenTheSection()
    Then I see that the current section is "Recent"                           # AppNavigationContext::iSeeThatTheCurrentSectionIs()
    Then I see that the file list contains a file named "Folder just created" # FileListContext::iSeeThatTheFileListContainsAFileNamed()
      Row for file Folder just created in file list could not be found after 100 seconds (NoSuchElementException)

  Scenario: show favorites                                            # /drone/src/github.com/nextcloud/server/tests/acceptance/features/app-files.feature:90
    Given I am logged in                                              # LoginPageContext::iAmLoggedIn()
    And I mark "welcome.txt" as favorite                              # FileListContext::iMarkAsFavorite()
    When I open the "Favorites" section                               # AppNavigationContext::iOpenTheSection()
    Then I see that the current section is "Favorites"                # AppNavigationContext::iSeeThatTheCurrentSectionIs()
    Then I see that the file list contains a file named "welcome.txt" # FileListContext::iSeeThatTheFileListContainsAFileNamed()
      Row for file welcome.txt in file list could not be found after 100 seconds (NoSuchElementException)

  Scenario: show deleted files                                        # /drone/src/github.com/nextcloud/server/tests/acceptance/features/app-files.feature:126
    Given I am logged in                                              # LoginPageContext::iAmLoggedIn()
    And I delete "welcome.txt"                                        # FileListContext::iDelete()
    When I open the "Deleted files" section                           # AppNavigationContext::iOpenTheSection()
    Then I see that the current section is "Deleted files"            # AppNavigationContext::iSeeThatTheCurrentSectionIs()
    Then I see that the file list contains a file named "welcome.txt" # FileListContext::iSeeThatTheFileListContainsAFileNamed()
      Row for file welcome.txt in file list could not be found after 100 seconds (NoSuchElementException)

TESTS=acceptance, TESTS-ACCEPTANCE=app-files-tags

tests/acceptance/features/app-files-tags.feature:11

Show full log

  Scenario: show the input field for tags in the details view after closing and opening the details view again # /drone/src/github.com/nextcloud/server/tests/acceptance/features/app-files-tags.feature:11
    Given I am logged in                                                                                       # LoginPageContext::iAmLoggedIn()
    And I open the details view for "welcome.txt"                                                              # FileListContext::iOpenTheDetailsViewFor()
    And I see that the details view is open                                                                    # FilesAppContext::iSeeThatTheDetailsViewIsOpen()
    And I close the details view                                                                               # FilesAppContext::iCloseTheDetailsView()
    And I see that the details view is closed                                                                  # FilesAppContext::iSeeThatTheDetailsViewIsClosed()
    And I open the details view for "welcome.txt"                                                              # FileListContext::iOpenTheDetailsViewFor()
    And I see that the details view is open                                                                    # FilesAppContext::iSeeThatTheDetailsViewIsOpen()
    When I open the input field for tags in the details view                                                   # FilesAppContext::iOpenTheInputFieldForTagsInTheDetailsView()
    Then I see that the input field for tags in the details view is shown                                      # FilesAppContext::iSeeThatTheInputFieldForTagsInTheDetailsViewIsShown()
      Failed asserting that false is true.

TESTS=acceptance, TESTS-ACCEPTANCE=header

tests/acceptance/features/header.feature:44

Show full log

  Scenario: search for other users in the contacts menu                              # /drone/src/github.com/nextcloud/server/tests/acceptance/features/header.feature:44
    Given I am logged in as the admin                                                # LoginPageContext::iAmLoggedInAsTheAdmin()
    And I open the User settings                                                     # SettingsMenuContext::iOpenTheUserSettings()
    And I click the New user button                                                  # UsersSettingsContext::iClickTheNewUserButton()
      New user button in Users Settings could not be found (NoSuchElementException)
    And I see that the new user form is shown                                        # UsersSettingsContext::iSeeThatTheNewUserFormIsShown()
    And I create user user1 with password 123456acb                                  # UsersSettingsContext::iCreateUserWithPassword()
    And I see that the list of users contains the user user1                         # UsersSettingsContext::iSeeThatTheListOfUsersContainsTheUser()
    And I open the Contacts menu                                                     # ContactsMenuContext::iOpenTheContactsMenu()
    And I see that the Contacts menu is shown                                        # ContactsMenuContext::iSeeThatTheContactsMenuIsShown()
    And I see that the contact "user0" in the Contacts menu is shown                 # ContactsMenuContext::iSeeThatTheContactInTheContactsMenuIsShown()
    And I see that the contact "user1" in the Contacts menu is shown                 # ContactsMenuContext::iSeeThatTheContactInTheContactsMenuIsShown()
    And I see that the Contacts menu search input is shown                           # ContactsMenuContext::iSeeThatTheContactsMenuSearchInputIsShown()
    When I search for the user "user0"                                               # ContactsMenuContext::iSearchForTheUser()
    Then I see that the contact "user1" in the Contacts menu is eventually not shown # ContactsMenuContext::iSeeThatTheContactInTheContactsMenuIsEventuallyNotShown()
    And I see that the contact "user0" in the Contacts menu is shown                 # ContactsMenuContext::iSeeThatTheContactInTheContactsMenuIsShown()
    And I see that the contact "admin" in the Contacts menu is not shown             # ContactsMenuContext::iSeeThatTheContactInTheContactsMenuIsNotShown()

TESTS=sqlite-php7.1-samba-non-native

Show full log

There was 1 failure:

1) OCA\Files_External\Tests\Storage\SmbTest::testNotifyGetChanges
Failed asserting that an array contains OC\Files\Notify\RenameChange Object &000000005be3f7190000000042e8657d (
    'targetPath' => 'renamed.txt'
    'type' => 4
    'path' => 'newfile.txt'
).

/drone/src/github.com/nextcloud/server/apps/files_external/tests/Storage/SmbTest.php:115

MorrisJobke · 2019-03-06T08:57:38Z

And there is the revert: #14557

Disable fragile tests for now

40e04c8

Signed-off-by: Morris Jobke <hey@morrisjobke.de>

MorrisJobke added the 3. to review Waiting for reviews label Mar 5, 2019

MorrisJobke added this to the Nextcloud 16 milestone Mar 5, 2019

MorrisJobke requested review from danxuliu, kesselb and rullzer March 5, 2019 13:26

MorrisJobke mentioned this pull request Mar 5, 2019

Check if password can be changed for the users backend in OCS api #14451

Merged

MorrisJobke requested review from ChristophWurst, nickvergessen and skjnldsv March 5, 2019 15:17

ChristophWurst approved these changes Mar 5, 2019

View reviewed changes

skjnldsv approved these changes Mar 5, 2019

View reviewed changes

MorrisJobke added 4. to release Ready to be released and/or waiting for tests to finish and removed 3. to review Waiting for reviews labels Mar 5, 2019

MorrisJobke merged commit 7fa0615 into master Mar 5, 2019

MorrisJobke deleted the ci/noid/disable-fragile-tests branch March 5, 2019 21:31

MorrisJobke mentioned this pull request Mar 6, 2019

Revert "Disable fragile tests for now" #14557

Closed

This was referenced Mar 6, 2019

16.0.0 Beta 1 #14573

Merged

Disable fragile tests #14640

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Disable fragile tests for now#14541

Disable fragile tests for now#14541
MorrisJobke merged 1 commit intomasterfrom
ci/noid/disable-fragile-tests

MorrisJobke commented Mar 5, 2019

Uh oh!

faily-bot bot commented Mar 5, 2019

Uh oh!

ChristophWurst left a comment

Uh oh!

MorrisJobke commented Mar 5, 2019

Uh oh!

MorrisJobke commented Mar 5, 2019

Uh oh!

faily-bot bot commented Mar 5, 2019

Uh oh!

MorrisJobke commented Mar 6, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

MorrisJobke commented Mar 5, 2019

Uh oh!

faily-bot bot commented Mar 5, 2019

Status of 16768: failure

Uh oh!

ChristophWurst left a comment

Choose a reason for hiding this comment

Uh oh!

MorrisJobke commented Mar 5, 2019

Uh oh!

MorrisJobke commented Mar 5, 2019

Uh oh!

faily-bot bot commented Mar 5, 2019

Status of 16789: failure

TESTS=acceptance, TESTS-ACCEPTANCE=app-comments

TESTS=acceptance, TESTS-ACCEPTANCE=app-files

TESTS=acceptance, TESTS-ACCEPTANCE=app-files-tags

TESTS=acceptance, TESTS-ACCEPTANCE=header

TESTS=sqlite-php7.1-samba-non-native

Uh oh!

MorrisJobke commented Mar 6, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants